An Extremely Goofy Movie

By Tim Monroe

Using Video Overrides and Tweening in Sprite Movies

Introduction

In the previous QuickTime Toolkit article ("A Goofy Movie" in MacTech, March 2001), we learned how to create some simple sprite movies. We saw that a sprite track typically contains two kinds of media samples, key frame samples and override samples. A key frame sample contains the set of images used by all the sprites in the track (up to the next key frame sample) and information about the initial properties of those sprites. An override sample contains only information about changes in the properties of the sprites. It's these changes that allow us to perform sprite animation with override samples.

In this article, we're going to investigate two ways to perform sprite animation without using override samples. We'll see how to use a video track as the source for a sprite's images, and we'll see how to interpolate a sequence of values for a sprite property. Given a starting value and an ending value, QuickTime is able to figure out, for any moment in the duration of the animation, what the appropriate value between those two values should be. This process is called tweening, and the track that contains the information needed to do the tweening is called a tween track.

Video override tracks and tween tracks are two kinds of modifier tracks, or tracks whose media data is used to modify data in some other track. Modifier tracks do not display their data directly in a movie. Rather, that data is used only to supplement or alter the data in some other track in the movie. A video override track supplements the data in a sprite track by providing a source of images for one or more sprites in that track. And a tween track can modify the data in a sprite track by providing a sequence of settings for one of the properties of a sprite in that track. For example, we can use a tween track to generate a sequence of horizontal positions for a sprite.

We'll begin by seeing how to use a video track as a source of image data for a sprite. Then we'll turn our attention to tweening. Tweening is an extremely useful technique throughout QuickTime, not just in connection with sprite properties. So it will be good to spend some time getting comfortable building tween tracks.

Our sample application this month builds on last month's QTSprites application, so I've called it QTSpritesPlus. Figure 1 shows the Test menu of QTSpritesPlus.

These menu items build movies that are modifications of the icon and penguin sprite movies that we built last time. The first menu item builds a movie that contains two sprites, and the image of one of those sprites is overridden by the frames from a video track. The next three menu items use tween tracks to perform different spatial animations on the icon sprite (moving it to the right, spinning it in place, and then both moving and spinning it at the same time). The last menu item builds the appearing-penguin movie once again, this time using a tween track to change the graphics mode of the penguin sprite image.

Video Override Tracks

It's actually quite simple to use a video track as the source for a sprite's images. We need to add a video track to our sprite movie, create a track reference from the sprite track to the video track, and then indicate to the sprite track how it is to interpret the data that it receives from the video track. Figure 2 shows our video override movie. Here, the sprite track contains two sprites; the image of one of those sprites is a picture of the Titanium PowerBook G4, and the image of the other sprite has been replaced by the frames of the video track. By properly setting the positions of both sprites and choosing a video track with just the right dimensions, we can get that video track to exactly overlay the screen of the PowerBook.

Building a Sprite Track

The first thing we need to do, of course, is build a new movie that contains a sprite track. In the present case, as just mentioned, we'll build a sprite track with a single key frame sample, which contains two sprites and two sprite images. What's different from the previous article is that we won't add any override frames to the sprite track. (In fact, if we were to add some override samples that change the sprite image index, we'd likely get some very strange results when we ran the movie. In technical terms, the results are undefined; in layman's terms, don't do it!) We'll set the duration of the key frame sample to 30 seconds (which is the duration of the video track we want to use as our image override track). Listing 1 shows the definition of the function QTSprites_AddPowerBookMovieSamplesToMedia, which we use to add the sample to the sprite track.

Listing 1: Adding a single key frame sample to a sprite track

QTSprites_AddPowerBookMovieSamplesToMedia

void QTSprites_AddPowerBookMovieSamplesToMedia 
                              (Media theMedia)
{
   QTAtomContainer         mySample = NULL;
   QTAtomContainer         mySpriteData = NULL;
   RGBColor                  myKeyColor;
   Point                     myLocation;
   short                     isVisible, myIndex, myLayer;
   OSErr                     myErr = noErr;

   // create a new, empty key frame sample
   myErr = QTNewAtomContainer(&mySample);
   if (myErr != noErr)
      goto bail;

   myKeyColor.red = myKeyColor.green = myKeyColor.blue = 
                              0xffff;      // white

   // add images to the key frame sample
   SpriteUtils_AddPICTImageToKeyFrameSample(mySample, 
               kOldQTIconID, &myKeyColor, 1, NULL, NULL);
   SpriteUtils_AddPICTImageToKeyFrameSample(mySample, 
            kTitaniumPowerBookID, &myKeyColor, 2, NULL, NULL);

   // add the initial sprite properties to the key frame sample
   myErr = QTNewAtomContainer(&mySpriteData);
   if (myErr != noErr)
      goto bail;

   // the QT icon sprite
   myLocation.h   = 46;
   myLocation.v   = 8;
   isVisible      = true;
   myIndex         = kOldQTIconImageIndex;
   myLayer         = 1;
   
   SpriteUtils_SetSpriteData(mySpriteData, &myLocation, 
               &isVisible, &myLayer, &myIndex, NULL, NULL, NULL);
   SpriteUtils_AddSpriteToSample(mySample, mySpriteData, 
               kQTIconSpriteAtomID);

   // the PowerBook sprite
   myLocation.h   = 0;
   myLocation.v   = 0;
   isVisible      = true;
   myIndex         = kPowerBookImageIndex;
   myLayer         = 2;

   SpriteUtils_SetSpriteData(mySpriteData, &myLocation, 
            &isVisible, &myLayer, &myIndex, NULL, NULL, NULL);
   SpriteUtils_AddSpriteToSample(mySample, mySpriteData, 
               kTitaniumPowerBookID);
   SpriteUtils_AddSpriteSampleToMedia(theMedia, mySample, 
               kSpriteMediaFrameDurationPowerBook, true, NULL);

bail:   
   if (mySample != NULL)
      QTDisposeAtomContainer(mySample);

   if (mySpriteData != NULL)
      QTDisposeAtomContainer(mySpriteData);
}

See last month's article for details on the sprite utilities (such as SpriteUtils_SetSpriteData) used in Listing 1.

Adding a Video Track

Now let's add a video track to the sprite movie. We'll do this by copying an existing video track from another movie into the sprite movie. We want the video track to be at least as long as the sprite track, so that there are enough frames to replace the sprite image for the entire duration of the sprite track. For simplicity, we'll make the video track exactly as long as the sprite track. Figure 3 shows the track layout that we want to achieve.

There are actually quite a number of ways to use QuickTime APIs to copy a video track from one movie to another. We could, for instance, iterate through the samples in the source video track and call AddMediaSample to copy them one by one into a new track in the sprite movie. Or we could select the entire source movie and then call the AddMovieSelection function to insert the selection into the sprite movie. Or we could use the InsertTrackSegment function to insert the entire source track into the sprite movie. That's the strategy we'll use in the QTSprites_ImportVideoTrack function, shown in Listing 2.

Listing 2: Copying a video track from one movie to another

QTSprites_ImportVideoTrack

OSErr QTSprites_ImportVideoTrack 
      (Movie theSrcMovie, Movie theDstMovie, Track *theTrack)
{
   Track                  mySrcTrack = NULL;
   Media                  mySrcMedia = NULL;
   Track                  myDstTrack = NULL;
   Media                  myDstMedia = NULL;
   Fixed                  myWidth, myHeight;
   OSErr                  myErr = paramErr;

   // get the first video track in the source movie
   mySrcTrack = GetMovieIndTrackType(theSrcMovie, 1, 
                              VideoMediaType, movieTrackMediaType);
   if (mySrcTrack == NULL)
      goto bail;

   // get the track's media and dimensions
   mySrcMedia = GetTrackMedia(mySrcTrack);
   GetTrackDimensions(mySrcTrack, &myWidth, &myHeight);

   // create a destination track
   myDstTrack = NewMovieTrack(theDstMovie, myWidth, myHeight, 
                              kNoVolume);
   if (myDstTrack == NULL)
      goto bail;

   // create a destination media
   myDstMedia = NewTrackMedia(myDstTrack, VideoMediaType, 
                     GetMediaTimeScale(mySrcMedia), 0, 0);
   if (myDstMedia == NULL)
      goto bail;

   myErr = BeginMediaEdits(myDstMedia);
   if (myErr != noErr)
      goto bail;

   myErr = CopyTrackSettings(mySrcTrack, myDstTrack);
   if (myErr != noErr)
      goto bail;

   myErr = InsertTrackSegment(mySrcTrack, myDstTrack, 0, 
                     GetTrackDuration(mySrcTrack), 0);
   if (myErr != noErr)
      goto bail;

   myErr = EndMediaEdits(myDstMedia);
   if (myErr != noErr)
      goto bail;

bail:
   if (theTrack != NULL)
      *theTrack = myDstTrack;

   return(myErr);
}

Notice that we call BeginMediaEdits and EndMediaEdits, so that the video track media samples are copied into the sprite movie. If we didn't call these two functions, the video track in the sprite movie would reference the samples in the source movie. (There's nothing intrinsically wrong with that, but we generally prefer to create self-contained movies.) Notice also that QTSprites_ImportVideoTrack returns the new track that it creates to the caller.

If our call to QTSprites_ImportVideoTrack returns successfully, we want to truncate the new video track so that it is exactly as long as the sprite track. We accomplish this by deleting any portion of the new video track that extends beyond the end of the sprite track, like this:

myDuration = GetMovieDuration(theSpriteMovie);

myErr = QTSprites_ImportVideoTrack(myVideoMovie, 
                              theSpriteMovie, &myVideoTrack);
if (myErr != noErr)
   goto bail;

DeleteMovieSegment(theSpriteMovie, myDuration, 
            GetMovieDuration(theSpriteMovie) - myDuration);

Now we need to establish a link between the video track and the sprite track to which it's going to send its data. We do this by creating a track reference from the sprite track to the video track. We first encountered track references in an earlier article ("Word Is Out", in MacTech, November 2000) when we learned how to create chapter tracks. In the present case, we want to add a track reference of type kTrackModifierReference from the sprite track to the new video track, like this:

myErr = AddTrackReference(theSpriteTrack, myVideoTrack, 
                              kTrackModifierReference, &myRefIndex);

When QuickTime sees that the video track is the target of a reference of type kTrackModifierReference, it knows that the video track is a modifier track and hence not to draw the video track in the movie box. Instead, it sends the video data to the track that contains the reference to the video track.

Setting the Input Map

But how does the sprite track know what to do with the data being sent to it from the video track? In particular, how does it know which sprite image is to be replaced by those video frames? This information is contained in a data structure called an input map that is attached to the track's media. A media's input map specifies how the track is to interpret any data being sent to it from a modifier track. In other words, whenever a movie contains a modifier track, then some other track needs to have an input map that tells the associated media handler what to do with the data from the modifier track. The track reference and the input map work together to link a modifier track to its target track and to specify how the data from the modifier track should modify the target track.

An input map is an atom container that contains one atom of type kTrackModifierInput for each modifier track that is sending data to the target track. It's perfectly possible that several modifier tracks each send their data to a particular target track. In that case, the target track's input map would contain several kTrackModifierInput atoms. The ID of each such atom must be set to the reference index returned by AddTrackReference when the track reference was created.

Each atom of type kTrackModifierInput in an input map must contain at least two child atoms. One of these children is always of type kTrackModifierType and specifies the kind of data the target track is going to receive from the modifier track; in the case of a video override track, the type of the modifier track input is kTrackModifierTypeImage. The file Movies.h defines constants for a large number of modifier input types, several of which we'll encounter later in this article:

enum {
   kTrackModifierTypeMatrix                     = 1,
   kTrackModifierTypeClip                        = 2,
   kTrackModifierTypeGraphicsMode            = 5,
   kTrackModifierTypeVolume                     = 3,
   kTrackModifierTypeBalance                  = 4,
   kTrackModifierTypeImage               = FOUR_CHAR_CODE('vide'),
   kTrackModifierObjectMatrix                  = 6,
   kTrackModifierObjectGraphicsMode         = 7,
   kTrackModifierType3d4x4Matrix            = 8,
   kTrackModifierCameraData                     = 9,
   kTrackModifierSoundLocalizationData      = 10,
   kTrackModifierObjectImageIndex            = 11,
   kTrackModifierObjectLayer                  = 12,
   kTrackModifierObjectVisible               = 13,
   kTrackModifierAngleAspectCamera            = 14,
   kTrackModifierPanAngle                  = FOUR_CHAR_CODE('pan '),
   kTrackModifierTiltAngle               = FOUR_CHAR_CODE('tilt'),
   kTrackModifierVerticalFieldOfViewAngle
                                                = FOUR_CHAR_CODE('fov '),
   kTrackModifierObjectQTEventSend      = FOUR_CHAR_CODE('evnt')
};

The type of the second child atom in an input map entry atom depends on the kind of data specified in the first child atom. For instance, with a video override track, it specifies the index of the sprite image that is to be replaced by the frames of the video track. Figure 4 shows the structure of the input map we'll use to attach a video override track to a sprite track.

Keep in mind that the sprite images in a key frame sample are stored in a list that is used by all the sprites in that track. This means that two or more sprites can have the same image (by having their image index property set to the same value). So it's possible that two or more sprites can have their image replaced by the frames of a single video track. Similarly, it's possible for a single sprite to get its image data from first one and then another video track. In that case, there would have to be several override video modifier tracks and several kTrackModifierInput atoms in the sprite track's input map.

Listing 3 shows the definition of the QTSprites_AddVideoEntryToInputMap function, which we use to add the appropriate children to an existing input map.

Listing 3: Adding a video override entry to an input map

QTSprites_AddVideoEntryToInputMap

OSErr QTSprites_AddVideoEntryToInputMap 
            (QTAtomContainer theInputMap, long theRefIndex, 
               long theID, OSType theType, char *theName)
{
#pragma unused(theName)
   QTAtom            myInputAtom;
   OSErr            myErr = noErr;

   // add an entry to the input map
   myErr = QTInsertChild(theInputMap, kParentAtomIsContainer, 
                  kTrackModifierInput, theRefIndex, 0, 0, NULL, 
                     &myInputAtom);
   if (myErr != noErr)
      goto bail;

   // add two child atoms to the parent atom;
   // these atoms define the type of the modifier input and the image index to override
   theType = EndianU32_NtoB(theType);
   myErr = QTInsertChild(theInputMap, myInputAtom, 
                     kTrackModifierType, 1, 0, sizeof(OSType), 
                     &theType, NULL);
   if (myErr != noErr)
      goto bail;

   theID = EndianS32_NtoB(theID);
   myErr = QTInsertChild(theInputMap, myInputAtom, 
                     kSpritePropertyImageIndex, 1, 0, sizeof(long), 
                     &theID, NULL);

bail:
   return(myErr);
}

In QTSpritesPlus, we create an input map by calling QTNewAtomContainer:

myErr = QTNewAtomContainer(&myInputMap);

We create a new atom container because we created the sprite track and therefore know that it doesn't have an input map yet. Alternatively, we can retrieve a media's existing input map by calling GetMediaInputMap, like so:

myErr = GetMediaInputMap(GetTrackMedia(theSpriteTrack), 
                              &myInputMap);

(In fact, it's probably preferable to call GetMediaInputMap always, since it will create a new input map for us if the specified media doesn't have one yet.) Next we need to call our function QTSprites_AddVideoEntryToInputMap to add the appropriate entries to the input map:

myErr = QTSprites_AddVideoEntryToInputMap(myInputMap, 
               myRefIndex, kOldQTIconImageIndex, 
               kTrackModifierTypeImage, NULL);

Finally, we attach the newly-configured input map to the sprite track media by calling SetMediaInputMap:

myErr = SetMediaInputMap(GetTrackMedia(theSpriteTrack), 
                     myInputMap);

At this point, we can dispose of the atom container that we created (or that GetMediaInputMap created for us):

QTDisposeAtomContainer(myInputMap);

Our complete function for adding a video override track to an existing sprite movie is shown in Listing 4.

Listing 4: Adding a video override track to a sprite movie

QTSprites_AddVideoOverrideTrack

OSErr QTSprites_AddVideoOverrideTrack 
                     (Movie theSpriteMovie, Track theSpriteTrack)
{
   Movie                        myVideoMovie = NULL;
   Track                        myVideoTrack = NULL;
   short                        myRefNum = kInvalidFileRefNum;
   short                        myResID = 0;
   FSSpec                        myFSSpec;
   OSType                      myTypeList[] = {kQTFileTypeMovie};
   short                        myNumTypes = 2;
   QTFrameFileFilterUPP      myFileFilterUPP = NULL;
   TimeValue                     myDuration;
   long                           myRefIndex;
   QTAtomContainer            myInputMap = NULL;
   OSErr                        myErr = noErr;

#if TARGET_OS_MAC
   myNumTypes = 0;
#endif

   // have the user select a file; make sure it has a video track in it
retry:
   myFileFilterUPP = QTFrame_GetFileFilterUPP
                              ((ProcPtr)QTFrame_FilterFiles);

   myErr = QTFrame_GetOneFileWithPreview(myNumTypes, 
                              (QTFrameTypeListPtr)myTypeList, 
                              &myFSSpec, myFileFilterUPP);
   if (myFileFilterUPP != NULL)
      DisposeNavObjectFilterUPP(myFileFilterUPP);

   if (myErr != noErr)
      goto bail;

   myErr = OpenMovieFile(&myFSSpec, &myRefNum, fsRdPerm);
   if (myErr != noErr)
      goto bail;

   // now fetch the first movie from the file
   myResID = 0;
   myErr = NewMovieFromFile(&myVideoMovie, myRefNum, &myResID, 
                     NULL, newMovieActive, NULL);
   if (myErr != noErr)
      goto bail;

   myVideoTrack = GetMovieIndTrackType(myVideoMovie, 1, 
                     VideoMediaType, movieTrackMediaType);
   if (myVideoTrack == NULL)
      goto retry;

   // copy the video track into the sprite movie
   myDuration = GetMovieDuration(theSpriteMovie);

   myErr = QTSprites_ImportVideoTrack(myVideoMovie, 
                     theSpriteMovie, &myVideoTrack);
   if (myErr != noErr)
      goto bail;

   // truncate the new video track to the length of the sprite movie
   DeleteMovieSegment(theSpriteMovie, myDuration, 
                  GetMovieDuration(theSpriteMovie) - myDuration);

   // attach the video track as a modifier to the sprite track
   
   // create a media input map
   myErr = QTNewAtomContainer(&myInputMap);
   if (myErr != noErr)
      goto bail;

   myErr = AddTrackReference(theSpriteTrack, myVideoTrack, 
                     kTrackModifierReference, &myRefIndex);
   if (myErr != noErr)
      goto bail;

   myErr = QTSprites_AddVideoEntryToInputMap(myInputMap, 
                     myRefIndex, kOldQTIconImageIndex, 
                     kTrackModifierTypeImage, NULL);
   if (myErr != noErr)
      goto bail;

   // attach the input map to the sprite track
   myErr = SetMediaInputMap(GetTrackMedia(theSpriteTrack), 
                     myInputMap);

bail:
   if (myVideoMovie != NULL)
      DisposeMovie(myVideoMovie);

   if (myRefNum != kInvalidFileRefNum)
      CloseMovieFile(myRefNum);

   if (myInputMap != NULL)
      QTDisposeAtomContainer(myInputMap);

   return(myErr);
}

So it really is fairly straightforward to use a video track as the source of a sprite's images. It's mostly just a matter of linking the video track and the sprite track in the correct manner, using a track reference and an input map. As we'll see shortly, we need to perform this same linkage between a sprite track and a tween track that's sending it data.

Tweening is the process of generating values that lie between two given values (an initial value and a final value) or that are in some other way algorithmically derived from some given data. When it was first introduced (in QuickTime version 2.5), the tween media handler supported only linear interpolation between initial and final values. That is to say, if (for instance) the initial and final values are integers, then the tweened values all lie on a straight line drawn between those two values, as illustrated in Figure 5. If the value at time 0 is, say, 10 and the value at time 30 is 30, then the tweened value at time 15 will be 20.

The tween media handler isn't limited to tweening only integer values, however. In QuickTime version 2.5, the tween media handler could work with a large number of types of data, defined by these constants:

enum {
   kTweenTypeShort                                 = 1,
   kTweenTypeLong                                 = 2,
   kTweenTypeFixed                                 = 3,
   kTweenTypePoint                                 = 4,
   kTweenTypeQDRect                              = 5,
   kTweenTypeQDRegion                              = 6,
   kTweenTypeMatrix                              = 7,
   kTweenTypeRGBColor                              = 8,
   kTweenTypeGraphicsModeWithRGBColor      = 9,
   kTweenType3dScale                              = '3sca',
   kTweenType3dTranslate                        = '3tra',
   kTweenType3dRotate                              = '3rot',
   kTweenType3dRotateAboutPoint               = '3rap',
   kTweenType3dRotateAboutAxis                  = '3rax',
   kTweenType3dQuaternion                        = '3qua',
   kTweenType3dMatrix                              = '3mat',
   kTweenType3dCameraData                        = '3cam',
   kTweenType3dSoundLocalizationData         = '3slc'
};

For instance, given two structures of type RGBColor, the tween media handler can generate a third RGBColor structure by interpolating the individual fields of the initial and final structures. In all the cases listed above, linear interpolation is used, although not all fields of a structure are always interpolated, as we'll see later. (We shall ignore the 3D tween types, as they are used to tween 3D tracks, which are not currently supported in Mac OS X.)

QuickTime 3.0 added support for another few handfuls of data types, defined by these constants:

enum {
   kTweenTypeQTFloatSingle               = 10,
   kTweenTypeQTFloatDouble               = 11,
   kTweenTypeFixedPoint                     = 12,
   kTweenTypePathToMatrixTranslation   = FOUR_CHAR_CODE('gxmt'),
   kTweenTypePathToMatrixRotation      = FOUR_CHAR_CODE('gxpr'),
   kTweenTypePathToMatrixTranslationAndRotation
                                                = FOUR_CHAR_CODE('gxmr'),
   kTweenTypePathToFixedPoint            = FOUR_CHAR_CODE('gxfp'),
   kTweenTypePathXtoY                        = FOUR_CHAR_CODE('gxxy'),
   kTweenTypePathYtoX                        = FOUR_CHAR_CODE('gxyx'),
   kTweenTypeAtomList                        = FOUR_CHAR_CODE('atom'),
   kTweenTypePolygon                        = FOUR_CHAR_CODE('poly'),
   kTweenTypeMultiMatrix                  = FOUR_CHAR_CODE('mulm'),
   kTweenTypeSpin                           = FOUR_CHAR_CODE('spin'),
   kTweenType3dMatrixNonLinear            = FOUR_CHAR_CODE('3nlr'),
   kTweenType3dVRObject                     = FOUR_CHAR_CODE('3vro')
};

These new tween types provide support for more complex tweening operations and for operations that are not simple linear interpolations of data. For instance, the various types of path tweens allow us to derive values based on the shape of an arbitrary curve defined by a vector path. And the list tween derives values from a list of atoms in an atom container, which can result in a series of discrete steps of non-continuous values.

In the remainder of this article, we'll investigate four of these tween types in detail: kTweenTypeGraphicsModeWithRGBColor, kTweenTypeMatrix, kTweenTypeSpin, and kTweenTypeMultiMatrix. Perhaps we'll return to consider some of the others in a future article.

Graphics Mode Tweening

Let's begin our hands-on work with tweening by reconsidering our old favorite, the appearing-penguin movie. In the previous article, we built a sprite version of this movie by creating a sprite track with one key frame sample (which holds the compressed penguin image and the initial sprite properties) and 99 override frames (which hold data of type ModifierTrackGraphicsModeRecord). We constructed the override samples so that the opacity of the sprite smoothly increases from total transparency to total opacity. This is a textbook case of where tweening can be of assistance. Instead of adding 99 override samples to the sprite track, we can instead add a single tween track to the movie that effectively says: start the graphics mode of the sprite image at total transparency and smoothly increase it up to total opacity.

A couple of pictures will help us appreciate the difference here. Figure 6 shows the original structure of the penguin sprite movie. Figure 7 shows the revised version, which uses a tween track in place of the override samples.

The start time and duration of the tween media sample determine the start time and duration of the tweening operation. As we'll see later, however, it's possible to limit the tweening to only part of the time spanned by the tween media sample.

Adding a Tween Track

We add a tween track to our sprite movie in the standard way, by calling NewMovieTrack and NewTrackMedia:

myTweenTrack = NewMovieTrack(theMovie, 0, 0, kNoVolume);
myTweenMedia = NewTrackMedia(myTweenTrack, TweenMediaType, 
                              GetMovieTimeScale(theMovie), NULL, 0);

Now we need to add a tween media sample to the tween track. A tween media sample is an atom container that contains one or more tween entries. A tween entry is an atom (of type kTweenEntry) that holds other atoms, which include at least a tween type atom (of type kTweenType) and a tween data atom (of type kTweenData). Figure 8 shows the general structure of a tween media sample.

The atom data in a tween type atom is the type of the tween atom (that is, one of the constants listed earlier). The kind of atom data in a tween data atom depends on the tween type; for instance, for a tween entry of type kTweenTypeGraphicsModeWithRGBColor, the atom data is a pair of ModifierTrackGraphicsModeRecord structures that specify the initial and final graphics modes. Listing 5 shows how we can build the tween media sample for our penguin movie.

Listing 5: Building a graphics mode tween media sample

QTSprites_AddTweenOverrideTrack

ModifierTrackGraphicsModeRecord   myGraphicsMode[2];
QTAtomContainer                           mySample = NULL;
QTAtom                                       myTweenEntryAtom = 0;

myErr = QTNewAtomContainer(&mySample);
if (myErr != noErr)
   goto bail;

// add a tween entry atom to the atom container
myErr = QTInsertChild(mySample, kParentAtomIsContainer, 
                  kTweenEntry, 1, 0, 0, NULL, &myTweenEntryAtom);
if (myErr != noErr)
   goto bail;

// set the type of this tween entry
myType = EndianU32_NtoB(kTweenTypeGraphicsModeWithRGBColor);
myErr = QTInsertChild(mySample, myTweenEntryAtom, kTweenType, 
                           1, 0, sizeof(myType), &myType, NULL);
if (myErr != noErr)
   goto bail;

// set the initial blend amount (0 = fully transparent)
myGraphicsMode[0].graphicsMode = EndianU32_NtoB(blend);
myGraphicsMode[0].opColor.red = 0;
myGraphicsMode[0].opColor.green = 0;
myGraphicsMode[0].opColor.blue = 0;

// set the final blend amount (0xffff = fully opaque)
myGraphicsMode[1].graphicsMode   = EndianU32_NtoB(blend);
myGraphicsMode[1].opColor.red      = EndianU16_NtoB(0xffff);
myGraphicsMode[1].opColor.green   = EndianU16_NtoB(0xffff);
myGraphicsMode[1].opColor.blue   = EndianU16_NtoB(0xffff);

myErr = QTInsertChild(mySample, myTweenEntryAtom, kTweenData, 
               1, 0, 2 * sizeof(ModifierTrackGraphicsModeRecord), 
                     myGraphicsMode, NULL);

As you can see, the first record specifies a fully transparent graphics mode, while the second specifies a fully opaque graphics mode. The tween media handler generates a series of records by interpolating the red, green, and blue fields. The graphicsMode field is not interpolated; rather, it's copied from the first record into the tweened record.

Now it's time to add the sample to the tween media. The tween media handler uses the generic sample description, of type SampleDescription. So we need to create a sample description, fill in the descSize field, and then call AddMediaSample, as shown in Listing 6.

Listing 6: Adding a sample to a tween media

QTSprites_AddTweenOverrideTrack

// create the sample description
mySampleDesc = (SampleDescriptionHandle)
                     NewHandleClear(sizeof(SampleDescription));
if (mySampleDesc == NULL)
   goto bail;

(**mySampleDesc).descSize = sizeof(SampleDescription);
      
// add the tween sample to the media
myErr = BeginMediaEdits(myTweenMedia);
if (myErr != noErr)
   goto bail;

myErr = AddMediaSample(myTweenMedia, mySample, 0, 
                  GetHandleSize(mySample), 
                  GetMediaDuration(GetTrackMedia(theTargetTrack)), 
                  (SampleDescriptionHandle)mySampleDesc, 
                  1, 0, NULL);
if (myErr != noErr)
   goto bail;

myErr = EndMediaEdits(myTweenMedia);

Notice that we specify the duration to AddMediaSample like this:

GetMediaDuration(GetTrackMedia(theTargetTrack))

This means that the tween media sample extends for the entire length of the sprite track (as shown in Figure 7). Finally, we need to insert the media into the tween track:

myErr = InsertMediaIntoTrack(myTweenTrack, 0, 0, 
                  GetMediaDuration(myTweenMedia), fixed1);

We've finished creating the tween track, but we still need to link it up with the sprite track so that when the movie is played back, the tween track sends its output (a record of type ModifierTrackGraphicsModeRecord) to the sprite track. The sprite track will use that output to set the graphics mode property of some sprite in the sprite track. We indicate both of these kinds of information (the type of modifier data the tween track is producing and the ID of the sprite to apply it to) in the sprite track's input map.

We create an input map in exactly the same way we did when working with a video override track, by calling QTNewAtomContainer:

myErr = QTNewAtomContainer(&myInputMap);

Then we need to add a track reference, from the target track (the sprite track) to the tween track:

myErr = AddTrackReference(theTargetTrack, myTweenTrack, 
                              kTrackModifierReference, &myRefIndex);

And then we need to add some data to the empty input map. QTSpritesPlus calls another application-defined function, QTSprites_AddTweenEntryToInputMap, passing in the input map, the track reference index obtained from AddTrackReference, the ID of the sprite to tween, and type of the modifier data:

myErr = QTSprites_AddTweenEntryToInputMap(myInputMap, 
                           myRefIndex, kPenguinSpriteAtomID, 
                           kTrackModifierObjectGraphicsMode, NULL);

QTSprites_AddTweenEntryToInputMap is defined in Listing 7. It's pretty much identical to QTSprites_AddVideoEntryToInputMap, except that the ID passed in now specifies the ID of the sprite to which the tween data is to be applied, not the image index of the sprite. Accordingly, the second child atom we add to the input map is of type kTrackModifierObjectID, not kSpritePropertyImageIndex.

Listing 7: Adding a tween entry to an input map

QTSprites_AddTweenEntryToInputMap

OSErr QTSprites_AddTweenEntryToInputMap 
               (QTAtomContainer theInputMap, long theRefIndex, 
               long theID, OSType theType, char *theName)
{
#pragma unused(theName)
   QTAtom            myInputAtom;
   OSErr            myErr = noErr;

   // add an entry to the input map
   myErr = QTInsertChild(theInputMap, kParentAtomIsContainer, 
                     kTrackModifierInput, theRefIndex, 0, 0, NULL, 
                     &myInputAtom);
   if (myErr != noErr)
      goto bail;

   // add two child atoms to the parent atom;
   // these atoms define the type of the modifier input and the ID of the sprite to       // receive the tween data
   theType = EndianU32_NtoB(theType);
   myErr = QTInsertChild(theInputMap, myInputAtom, 
                     kTrackModifierType, 1, 0, sizeof(OSType), 
                     &theType, NULL);
   if (myErr != noErr)
      goto bail;

   theID = EndianU32_NtoB(theID);
   myErr = QTInsertChild(theInputMap, myInputAtom, 
                     kTrackModifierObjectID, 1, 0, sizeof(long), 
                     &theID, NULL);

bail:
   return(myErr);
}

Finally, we need to set the sprite track's input map, by calling SetMediaInputMap:

myErr = 
      SetMediaInputMap(GetTrackMedia(theTargetTrack), 
                           myInputMap);

Voilà, we've just created our first tween track and linked it to the sprite track. If we save the movie to disk (by calling AddMovieResource and CloseMovieFile) and then reopen and play it, we'll see the same sequence of frames that we saw when playing the original penguin movie or the original sprite version of the penguin movie. There are, however, several important advantages to using a tween track in place of sprite track override samples. First of all, since we have stored only two ModifierTrackGraphicsModeRecord structures instead of 99, we can expect to see some reduction in the size of the movie file. In fact, the size of the penguin movie file is reduced from 36 kilobytes to 28 kilobytes (almost one-quarter smaller).

The second important advantage to using a tween track is that it has no pre-established frame rate. In the override sample version of the penguin movie, we'll get at most 10 frames per second, because we have 100 frames in a 10-second movie. (I say "at most" 10 frames per second because QuickTime might need to drop some frames if the user's computer is not capable of displaying 10 frames per second.) With the tween track version, we'll get as many frames per second as QuickTime and the accompanying hardware can manage — which should easily surpass 10 frames per second. (Indeed, on a mid-range PowerMac G3, I clocked the tween version of the penguin movie at a brisk 83 frames per second!) So when, in the previous paragraph, I said that we'll see "the same sequence of frames", I was fibbing; in fact we're likely to get a much better visual output with the tween track movie than with the override sample movie. Smaller and better; ain't life grand?

Matrix Tweening

Now that you're convinced that tween tracks are worth playing with, let's consider a few more examples. In the previous article, we changed the horizontal position of the icon sprite by adding a bunch of override samples, each of which contained a matrix record specifying a new position. Once again, we can replace all those override samples with a single tween track, which contains the initial and final matrices. At run time, the tween media handler generates the intermediate matrices and funnels them to the sprite media handler, which applies them to the sprite to achieve the horizontal motion.

Building the Tween Data Atom

Listing 8 shows the code that we use to build the tween data atom in the media sample for the tween track. The atom contains two matrices, one for the initial position of the icon sprite and a second for its final position. Note that the matrices are just concatenated together as the atom data, not inserted into separate atoms.

Listing 8: Building a matrix tween media sample

QTSprites_AddTweenOverrideTrack

MatrixRecord         myMatrix[2];

// set the initial data for this tween entry
SetIdentityMatrix(&myMatrix[0]);
TranslateMatrix(&myMatrix[0], 
               Long2Fix(kIconDimension + (kIconDimension / 2)), 
               Long2Fix(kIconDimension + (kIconDimension / 2)));
EndianUtils_MatrixRecord_NtoB(&myMatrix[0]);

// set the final data for this tween entry
SetIdentityMatrix(&myMatrix[1]);
TranslateMatrix(&myMatrix[1], 
               Long2Fix(230 + (kIconDimension / 2)), 
               Long2Fix(kIconDimension + (kIconDimension / 2)));
EndianUtils_MatrixRecord_NtoB(&myMatrix[1]);

myErr = QTInsertChild(mySample, myTweenEntryAtom, kTweenData, 
               1, 0, 2 * sizeof(MatrixRecord), myMatrix, NULL);

The tween track is built exactly as above; the only difference here is that the modifier input type is kTrackModifierObjectMatrix. Once again, the resulting movie file is smaller than the version that contains override samples. Also, thanks to the increase in frames per second, the movie playback is noticeably smoother than the override samples version.

Setting the Tween Offset and Duration

As we've seen, a tween operation begins at the start of a tween media sample and extends for the entire length of the sample. It's possible, however, to add a couple of atoms to a tween entry atom to modify this default behavior.

If we want a tweening operation to begin at some time after the start of a media sample, we can include a tween offset atom, of type kTweenStartOffset. The data for a tween offset atom is a value of type TimeValue that indicates how far into the tween media sample the tweening operation is to begin. (This value should be specified in the media's time scale.) If theSample is a tween media sample atom container and myAtom is a tween entry atom, then we can add a tween offset atom to that tween entry atom like this:

theOffset = EndianS32_NtoB(theOffset);
myErr = QTInsertChild(theSample, myAtom, kTweenStartOffset, 
                     1, 1, sizeof(TimeValue), &theOffset, NULL);

The index and ID of the tween offset atom must both be 1. If we want to modify the duration of a tweening operation, we can add a tween duration atom, of type kTweenDuration, to a tween entry atom. The data for a tween duration atom is a value of type TimeValue. We can add a tween duration atom to a tween entry atom like this:

theDuration = EndianS32_NtoB(theDuration);
myErr = QTInsertChild(theSample, myAtom, kTweenDuration, 
                     1, 1, sizeof(TimeValue), &theDuration, NULL);

Once again, the index and ID of the tween duration atom must both be 1. The file QTSpritesPlus.c defines two functions, QTSprites_SetTweenEntryStartOffset and QTSprites_SetTweenEntryDuration, that you can use to set a tween's offset and duration.

Spin Tweening

Suppose now that we want a sprite to spin around in the sprite track. The standard way to get a sprite to spin is to include a large number of sprite images in the key frame sample and to change the sprite's image index during playback. (See the "space movie" created by last month's sample application.) Or, we could try to use the matrix tween just discussed, in conjunction with the RotateMatrix function, to set up a rotation matrix for the sprite. It turns out, however, that this is trickier than you'd suspect. If we wanted to rotate the icon one full turn, we might think we could specify a final matrix like this:

SetIdentityMatrix(&myMatrix[1]);
RotateMatrix(&myMatrix[1], Long2Fix(360), 0, 0);

But that won't do at all; 360ö is the same position as 0ö, so the resulting tween would effectively do nothing. Worse yet, it's not clear how we'd specify rotating two or more times. No doubt we could chop things up into a number of smaller rotations, but that's getting unduly messy.

To allow us to easily spin an object an arbitrary number of rotations, QuickTime 3.0 introduced spin tweening. The tween atom data for a spin tween consists of two Fixed values, the initial rotation amount and the total number of rotations. Listing 9 shows part of our code for building a tween track sample that spins the QuickTime icon kNumRotations times (which is defined in QTSpritesPlus.h as 5).

Listing 9: Building a spin tween media sample

QTSprites_AddTweenOverrideTrack

Fixed         mySpinData[2];

// set the initial rotation value
mySpinData[0] = EndianU32_NtoB(0);

// set the number of rotations
mySpinData[1] = EndianU32_NtoB(Long2Fix(kNumRotations));

myErr = QTInsertChild(mySample, myTweenEntryAtom, kTweenData, 
               1, 0, 2 * sizeof(Fixed), mySpinData, NULL);

The output of a spin tweening operation is a matrix (of type MatrixRecord) that can be applied to a sprite to achieve the desired rotation. The point of rotation (that is, the point about which the sprite image is rotated) is the image's registration point, which is set at the time the sprite image is added to the key frame. Note that the registration point is not a sprite property and cannot therefore be changed dynamically. Rather, the registration point is a property of the sprite image. The default registration point is (0, 0), or the upper-left corner of the box surrounding the sprite image.

Figure 9 shows a sprite movie in which the QuickTime icon rotates about the default registration point. Notice that the icon is now located at track position (0, 0). That's because the matrix generated by the spin tween overrides the matrix in the key frame sample (with which we specified the desired position of the icon).

When we run the movie, the icon spins around the point (0, 0), which means that it keeps moving in and out of the movie box. Now that's goofy.

Let's move the registration point of the icon image to the center of the image, like so:

myPoint.x = Long2Fix(kIconDimension / 2);
myPoint.y = Long2Fix(kIconDimension / 2);

// add images to the key frame sample
SpriteUtils_AddPICTImageToKeyFrameSample(mySample, 
                  kOldQTIconID, &myKeyColor, 1, &myPoint, NULL);

This gives us a movie that's slightly better, but still not wholly satisfactory. (See Figure 10.) The icon is still stuck in the upper-left corner of the movie box. What we'd like, I think, is for the icon to be situated at some other location in the movie box and spin around its registration point at that location.

Multimatrix Tweens

What we need is to be able to generate a matrix that both translates and spins the sprite image. This is a job for the multimatrix tween, introduced in QuickTime 3.0. A multimatrix tween concatenates two or more matrices produced by other kinds of tweens. To create a multimatrix tween, we build the atoms for the individual tweens and then add them to the data atom of a multimatrix tween entry. Figure 11 shows the atom structure that we want to build.

Notice that the tween data atom of the multimatrix tween entry is a parent atom that contains two tween entry atoms, one for the tween that spins the sprite image and one for the tween that translates the sprite image.

Listing 10 gives the code we use to add atoms to a parent atom (myTweenEntryAtom) to build a multimatrix tween atom. Matrix operations are not in general commutative, so the order in which we add the tween entry atoms for the matrix tweens to the multimatrix tween data atom is important. In Listing 10, we add the spin tween and then the translation tween.

Listing 10: Building a multimatrix tween media sample

QTSprites_AddTweenOverrideTrack

QTAtom                  myMultiTweenDataAtom = 0;
QTAtom                  myAtom = 0;
MatrixRecord         myMatrix[2];
Fixed                  mySpinData[2];

// add a multimatrix tween data atom to the tween entry atom
myErr = QTInsertChild(mySample, myTweenEntryAtom, kTweenData, 
               1, 0, 0, NULL, &myMultiTweenDataAtom);
if (myErr != noErr)
   goto bail;
         
// add a spin tween to the multimatrix tween data atom
myErr = QTInsertChild(mySample, myMultiTweenDataAtom, 
               kTweenEntry, 1, 0, 0, NULL, &myAtom);
if (myErr != noErr)
   goto bail;

// set the type of this tween entry
myType = EndianU32_NtoB(kTweenTypeSpin);
myErr = QTInsertChild(mySample, myAtom, kTweenType, 1, 0, 
               sizeof(myType), &myType, NULL);
if (myErr != noErr)
   goto bail;

// set the initial rotation value
mySpinData[0] = EndianU32_NtoB(0);

// set the number of rotations
mySpinData[1] = EndianU32_NtoB(Long2Fix(kNumRotations));

myErr = QTInsertChild(mySample, myAtom, kTweenData, 1, 0, 
               2 * sizeof(Fixed), mySpinData, NULL);
if (myErr != noErr)
   goto bail;

// add a translation matrix tween to the multimatrix tween data atom
myErr = QTInsertChild(mySample, myMultiTweenDataAtom, 
               kTweenEntry, 2, 0, 0, NULL, &myAtom);
if (myErr != noErr)
   goto bail;

// set the type of this tween entry
myType = EndianU32_NtoB(kTweenTypeMatrix);
myErr = QTInsertChild(mySample, myAtom, kTweenType, 1, 0, 
               sizeof(myType), &myType, NULL);
if (myErr != noErr)
   goto bail;

// set the initial data for this tween entry
SetIdentityMatrix(&myMatrix[0]);
TranslateMatrix(&myMatrix[0], 
               Long2Fix(kIconDimension + (kIconDimension / 2)), 
               Long2Fix(kIconDimension + (kIconDimension / 2)));
EndianUtils_MatrixRecord_NtoB(&myMatrix[0]);

// set the final data for this tween entry
SetIdentityMatrix(&myMatrix[1]);
TranslateMatrix(&myMatrix[1], 
               Long2Fix(230 + (kIconDimension / 2)), 
               Long2Fix(kIconDimension + (kIconDimension / 2)));
EndianUtils_MatrixRecord_NtoB(&myMatrix[1]);

myErr = QTInsertChild(mySample, myAtom, kTweenData, 1, 0, 
               2 * sizeof(MatrixRecord), myMatrix, NULL);

Conclusion

Video override tracks and tween tracks provide two different ways for us to enhance our sprite movies. Video override tracks give us a way to tap into an existing video track as a source of images for a sprite; they are kind of a one-trick pony, but nonetheless useful in the right circumstances. Tween tracks, by contrast, are quite generally useful; they give us a way to algorithmically alter sprite properties without using override samples. Tween tracks have the added benefits of smaller file sizes and increased frame rates. They are also usually easier to construct than a sequence of override samples. Why, after all, should we bother to do the math to figure out how to fade from full transparency to full opacity? Let's just let the tween media handler do it for us.

In the next article, we're going to consider yet another way to enhance our sprite movies, by attaching wired actions to the sprites. Our goal is not only to make our sprites active (using override samples or video override tracks or tween tracks), but also to make them interactive.

Tim Monroe is a member of the QuickTime engineering team. You can contact him at monroe@apple.com.