Volume Number: 17 (2001)
Issue Number: 12
Column Tag: Quicktime Toolkit

by Tim Monroe

Captured

Using Sequence Grabber Components to Capture Video and Sound

Introduction

From its inception, QuickTime has included the ability to capture video and sound data from devices attached to a computer. Capturing video and sound together involves attaching a camera or other device that provides an audiovisual data stream to the computer. These devices include camcorders, laserdisc players, televisions, and videocassette recorders. We can capture sound alone using an internal or external microphone, or by attaching a CD player or other sound-only device to the computer.

On the Macintosh models available in the early days of QuickTime, special add-on hardware (usually in the form of a NuBus or PCI card) was required to digitize an analog stream from an external device. Some models, beginning with the Power Macintosh 6100AV, included built-in audiovisual hardware that allowed the user to connect external devices to RCA-type or S-video connectors. Nowadays all Macintosh computers (and many Windows computers) support FireWire connections, which allow a pure digital stream of audiovisual data to be sent to the computer from camcorders and other devices.

At the lowest level, QuickTime interacts with video and sound hardware attached to a computer through software modules called video digitizer components (or just video digitizers) and sound input device drivers. A video digitizer digitizes the video data stream, if necessary, and often provides additional services such as resizing the video, clipping out portions of the video, and converting colors in the video. A sound input device driver manages communications between applications and the sound input hardware.

Normally, however, we won’t work with video digitizers or sound input device drivers directly. Instead, we’ll work with a sequence grabber component, a part of QuickTime that provides a set of high-level APIs for capturing video and sound data. Since there is virtually always just one available sequence grabber component, we’ll usually talk about the sequence grabber. The sequence grabber insulates us from having to know about any of the low-level details of video digitizers and sound input device drivers, and provides some additional services as well. We can use the sequence grabber to display video in a window, capture individual frames of video as pictures, and capture sequences of video frames as QuickTime movies. The sequence grabber can also capture sound data and synchronize the video and sound streams when displaying them in a window or capturing them to a movie file.

In this article, we’re going to see how to use the sequence grabber to capture video and sound data. We’ll develop an application, called QTCapture, which can capture video and sound from any available devices. The Test menu of QTCapture (on Windows, for a change) is shown in Figure 1.

Figure 1: The Test menu of QTCapture

The first two menu items display dialog boxes that allow us to configure the video and sound capture settings. For instance, we can set the video or sound source (the device from which we want to capture data) and the desired compression to be applied to the captured data. The third and fourth menu items allow us to specify whether we want to capture video, sound, or both. Using the “Split Track Files” menu item, we can specify that the captured video and sound data be written to different files. (By default, the sequence grabber writes the video and sound data into the same output file.) The next block of menu items allows us to select the size of the monitor window, the window in which the incoming video stream is displayed. Figure 2 shows QTCapture’s monitor window at its default size.

Figure 2: The monitor window of QTCapture

We use the last menu item to begin recording data to a file; in QTCapture, the recording stops when the user clicks the mouse button.

We’ll begin by taking a look at the sequence grabber and where it fits into the QuickTime architecture. Then we’ll see how to monitor the captured data, adjust the capture settings, and write the captured data into a file.

Sequence Grabber Overview

The sequence grabber is a part of QuickTime that can be used to monitor video and sound sources, capture images and sequences of images, and synchronize captured sound and video. The sequence grabber provides two main services to applications: previewing and recording. To preview a data source is to display the captured data in a window on the screen (if it’s visual data) or to play back the captured data through the sound output hardware (if it’s audio data). To record a data source is to write the captured data into one or more files on disk.

In QuickTime version 2.5, the sequence grabber gained the ability to capture text data, using text digitizer components. A text digitizer component captures text data from external sources, such as the closed-captioned data embedded in some television broadcasts. The process of capturing text is entirely analogous to the process of capturing video or sound, and it would be easy to extend our sample application QTCapture to also capture text data and record it into a text track in a QuickTime movie. Because very few computers are equipped to capture text data, however, we won’t consider the sequence grabber’s text capturing abilities further.

A sequence grabber component does not communicate directly with either a video digitizer or a sound input device driver. Instead, it communicates with one or more sequence grabber channel components. The sequence grabber channel components, in turn, communicate with the video digitizer components and sound input device drivers. Channel components send control information to them and receive digitized data from them; the digitized data is then passed to other parts of QuickTime for previewing or recording.

A sequence grabber component is also responsible for displaying any dialog boxes required to elicit capture settings from the user, such as the video settings dialog box shown in Figure 3. To do this, a sequence grabber component calls a sequence grabber panel component. A panel component then communicates with a channel component or the digitizer component to get and set the capture settings.

Figure 3: The video settings dialog box

Opening the Sequence Grabbing Components

The QuickTime capture architecture may seem fairly complicated, but in practice our applications need to work directly with only three components: a sequence grabber component and two sequence grabber channel components. QTCapture permits only one preview or record operation at a time, so it uses some global variables to keep track of these three components:

SeqGrabComponent         gSeqGrabber = NULL;
SGChannel                  gVideoChannel = 0;
SGChannel                  gSoundChannel = 0;

We’ll open an instance of a sequence grabber component by calling the OpenDefaultComponent function, like this:

gSeqGrabber = OpenDefaultComponent(SeqGrabComponentType, 0);

We then need to initialize this component by calling the SGInitialize function:

myErr = SGInitialize(gSeqGrabber);

SGInitialize allocates any additional memory the sequence grabber may need and performs other necessary set-up for subsequent previewing and recording.

Since we are going to be previewing video data, we need to tell the sequence grabber where to draw the previewed data. We do this by calling the SGSetGWorld function. With QTCapture, our monitor window is simply a dialog box, which we open like this:

gMonitor = GetNewDialog(kMonitorDLOGID, NULL, 
            (WindowPtr)-1L);

If we successfully open this dialog box and initialize the sequence grabber, we can then set the sequence grabber’s graphics world by calling SGSetGWorld:

myErr = SGSetGWorld(gSeqGrabber, GetDialogPort(gMonitor), 
            NULL);

It’s even easier to open the two sequence grabber channel components we need; we just call SGNewChannel, passing in the media type of the data to be captured:

SGNewChannel(gSeqGrabber, VideoMediaType, &gVideoChannel);
SGNewChannel(gSeqGrabber, SoundMediaType, &gSoundChannel);

Our actual code, of course, checks the result codes returned by SGNewChannel.

Configuring Video Channels

Before we can begin previewing or recording from these channels, we need to do some preliminary configuration of the channels and of our application. The first thing we want to do is set the channel usage flags of the video channel. These flags tell the channel component what operations we’re going to want it to perform. Currently, these channel usage flags are defined (in the file QuickTimeComponents.h):

enum {
   seqGrabRecord                        = 1,
   seqGrabPreview                        = 2,
   seqGrabPlayDuringRecord            = 4
};

These flags are fairly self-explanatory. The seqGrabRecord and seqGrabPreview flags tell a sequence grabber channel component that its channel will be used for recording and previewing, respectively. The seqGrabPlayDuringRecord flag indicates that we are going to want to preview the captured data while we are recording it. The previewed video will get choppier if we enable this flag (since we’re devoting some processor time to recording), but at least it will continue playing.

In QTCapture, we want to enable all of these flags for the video channel. So we’ll call SGSetChannelUsage like this:

myErr = SGSetChannelUsage(gVideoChannel, 
   seqGrabPreview | seqGrabRecord | seqGrabPlayDuringRecord);

We also want to set the initial size of our monitor window to its default size, which is half the size of the video digitizer’s active source rectangle (that is, the portion of the digitizer’s source rectangle that actually contains video data). We call the SGGetSrcVideoBounds function to get the size of the active source rectangle and then we resize our monitor window to half that size, using the code in Listing 1. (We’ll see the complete definition of QTCap_Init later, in Listing 3.)

Listing 1: Setting the initial size of the monitor window

QTCap_Init

short      myWidth;
short      myHeight;

myErr = SGGetSrcVideoBounds(gVideoChannel, 
            &gActiveVideoRect);
if (myErr == noErr) {
   myWidth = (gActiveVideoRect.right – gActiveVideoRect.left) 
                        / 2;
   myHeight = (gActiveVideoRect.bottom – gActiveVideoRect.top) 
                        / 2;
   SizeWindow(GetDialogWindow(gMonitor), myWidth, myHeight, 
                        false);
}

The last thing we need to do is tell the channel component the size of the display boundary rectangle, which is the rectangle in which the previewed video data is to be displayed. We can do this by retrieving the current size of the monitor window’s content region and then passing that size to the sequence grabber channel component by calling SGSetChannelBounds, like so:

GetPortBounds(GetDialogPort(gMonitor), &myRect);
myErr = SGSetChannelBounds(gVideoChannel, &myRect);

If any of this configuring should fail, then we won’t be able to capture or preview data from the video source. In that case, we want to close down the video channel and set the global variable gVideoChannel to NULL, indicating that we don’t have an open video channel:

if (myErr != noErr) {
   SGDisposeChannel(gSeqGrabber, gVideoChannel);
   gVideoChannel = NULL;
}

Configuring Audio Channels

Our audio channel is somewhat easier to configure. First, we want to set the channel usage, like this:

myErr = SGSetChannelUsage(gSoundChannel, 
   seqGrabPreview | seqGrabRecord);

You’ll notice that we did not set the seqGrabPlayDuringRecord flag. This makes good sense, since we don’t want the channel’s sound data to be played while it’s being recorded. On the other hand, we do want the sound to be played while it’s being previewed. Even in that case, however, we want to make sure that the volume of the sound played back is low enough to avoid any feedback that might arise if the sound input hardware (usually, the microphone) happens to be too near the speakers. So we’ll call SGSetChannelVolume to set the sound channel volume to a fairly low setting:

myErr = SGSetChannelVolume(gSoundChannel, 0x0010);

One other thing we want to do is add some sample rates to the Rate pop-up menu in the sound settings dialog box. By default, the only rates that appear in that menu are those that the underlying sound hardware indicates it can handle natively. On most modern Macintosh computers, for instance, only the 44.1 kHz rate appears (as seen in Figure 4), and on slightly older models only the 44.1 and 22.050 kHz rates appear.

Figure 4: The default Rate pop-up menu

The sequence grabber provides the SGSetAdditionalSoundRates function, which we can use to add some more rates to that menu. Listing 2 shows the code we use to add another 5 common sound sample rates to the Rate pop-up menu. The expanded menu is shown in Figure 5.

Listing 2: Adding sample rates to the sound settings dialog box

QTCap_Init

Handle      myRates = NULL;

myRates = NewHandleClear(5 * sizeof(Fixed));
if (myRates != NULL) {
   *((long *)(*myRates) + 0) = Long2Fix(8000);   // 8kHz
   *((long *)(*myRates) + 1) = Long2Fix(11025);  // 11kHz
   *((long *)(*myRates) + 2) = Long2Fix(16000);  // 16kHz
   *((long *)(*myRates) + 3) = Long2Fix(22050);  // 22kHz
   *((long *)(*myRates) + 4) = Long2Fix(32000);  // 32kHz
   SGSetAdditionalSoundRates(gSoundChannel, myRates);

   DisposeHandle(myRates);
}

Figure 5: The expanded Rate pop-up menu

Once again, if any of this configuring should fail, we want to close down the sound channel and set the global variable gSoundChannel to NULL:

if (myErr != noErr) {
   SGDisposeChannel(gSeqGrabber, gSoundChannel);
   gSoundChannel = NULL;
}

Previewing

Let’s reflect on what we’ve accomplished so far. We’ve opened an instance of the sequence grabber component. We’ve also opened two sequence grabber channels — one for video and one for sound — and we’ve configured both of those channels. We’ve also opened our monitor window and resized it to its default size. We haven’t yet displayed the monitor window, however, so let’s do that now:

MacShowWindow(GetDialogWindow(gMonitor));

All that remains, then, is to start the previewing. We can do that with a single call:

myErr = SGStartPreview(gSeqGrabber);

We also need to make sure that the sequence grabber gets some processor time periodically. We do that by calling SGIdle fairly often. In QTCapture, we’ll insert these lines of code into the application function QTApp_Idle:

if (gSeqGrabber != NULL)
   SGIdle(gSeqGrabber);

And we’re done! The application will display the captured video in the monitor window and play the captured sound through the computer’s speakers. Listing 3 shows the complete definition of the QTCap_Init function, which performs all the necessary set-up and then starts the preview rolling.

Listing 3: Initializing and the sequence grabber

QTCap_Init

ComponentResult QTCap_Init (void)
{
   ComponentResult            myErr = noErr;

   // open the sequence grabber component
   gSeqGrabber = OpenDefaultComponent(SeqGrabComponentType, 
            0);
   if (gSeqGrabber == NULL) {
      myErr = cantOpenHandler;
      goto bail;
   }

   // open the monitor window
   gMonitor = GetNewDialog(kMonitorDLOGID, NULL, 
            (WindowPtr)-1L);
   if (gMonitor == NULL) {
      myErr = memFullErr;
      goto bail;
   }

   SetPortDialogPort(gMonitor);
   MacMoveWindow(GetDialogWindow(gMonitor), 10, 
            30 + GetMBarHeight(), 0);

   // initialize the sequence grabber
   myErr = SGInitialize(gSeqGrabber);
   if (myErr == noErr) {
      // configure the sequence grabber component
      myErr = SGSetGWorld(gSeqGrabber, GetDialogPort(gMonitor), 
            NULL);
      if (myErr != noErr)
         goto bail;

      // create a video channel
      myErr = SGNewChannel(gSeqGrabber, VideoMediaType, 
            &gVideoChannel);
      if ((gVideoChannel != NULL) && (myErr == noErr)) {
         short      myWidth;
         short      myHeight;
         Rect         myRect;

         myErr = SGGetSrcVideoBounds(gVideoChannel, 
            &gActiveVideoRect);
         if (myErr == noErr) {
            myWidth = (gActiveVideoRect.right – 
                                 gActiveVideoRect.left) / 2;
            myHeight = (gActiveVideoRect.bottom – 
                                 gActiveVideoRect.top) / 2;
            SizeWindow(GetDialogWindow(gMonitor), myWidth, 
                                 myHeight, false);
         }

         myErr = SGSetChannelUsage(gVideoChannel, 
                              seqGrabPreview | seqGrabRecord | 
                              seqGrabPlayDuringRecord);
         if (myErr == noErr) {
            GetPortBounds(GetDialogPort(gMonitor), &myRect);
            myErr = SGSetChannelBounds(gVideoChannel, &myRect);
         }

         // if an error occurred while configuring video channel, dispose of it
         if (myErr != noErr) {
            SGDisposeChannel(gSeqGrabber, gVideoChannel);
            gVideoChannel = NULL;
         }
      }

      // create a sound channel
      myErr = SGNewChannel(gSeqGrabber, SoundMediaType, 
            &gSoundChannel);
      if ((gSoundChannel != NULL) && (myErr == noErr)) {
         Handle      myRates = NULL;

         myErr = SGSetChannelUsage(gSoundChannel, 
            seqGrabPreview | seqGrabRecord);
         if (myErr == noErr) {
            // set the volume low to prevent feedback when we start the preview
            // (in case the mic is anywhere near the speaker)
            myErr = SGSetChannelVolume(gSoundChannel, 0x0010);
         }

         // add some sample rates to the Sound settings dialog box Rate pop-up menu
         myRates = NewHandleClear(5 * sizeof(Fixed));
         if (myRates != NULL) {
            *((long *)(*myRates) + 0) = Long2Fix(8000);   // 8kHz
            *((long *)(*myRates) + 1) = Long2Fix(11025);  // 11kHz
            *((long *)(*myRates) + 2) = Long2Fix(16000);  // 16kHz
            *((long *)(*myRates) + 3) = Long2Fix(22050);  // 22kHz
            *((long *)(*myRates) + 4) = Long2Fix(32000);  // 32kHz
            SGSetAdditionalSoundRates(gSoundChannel, myRates);

            DisposeHandle(myRates);
         }

         // if an error occurred while configuring sound channel, dispose of it
         if (myErr != noErr) {
            SGDisposeChannel(gSeqGrabber, gSoundChannel);
            gSoundChannel = NULL;
         }
      }
   }

   // display the monitor window
   MacShowWindow(GetDialogWindow(gMonitor));

   // start previewing
   if (myErr == noErr)
      myErr = SGStartPreview(gSeqGrabber);

bail:
   // if an error occurred, clean up
   if (myErr != noErr)
      QTCap_Stop();

   return(myErr);
}

We call QTCap_Init when QTCapture starts up, so that the monitor window appears immediately at application start-up time. Our menu-adjusting function QTApp_AdjustMenus contains these lines, which disable the “Close” menu item in the File menu if the monitor window is the frontmost window:

if (QTFrame_GetFrontAppWindow() == 
            QTFrame_GetWindowReferenceFromWindow
                                             (GetDialogWindow(gMonitor)))
   QTFrame_SetMenuItemState(myMenu, IDM_FILECLOSE, 
               kDisableMenuItem);

So the monitor window will remain open for as long as QTCapture is running. When QTCapture quits, we close the monitor window and shut down our sequence grabber components by calling the QTCap_Stop function (defined in Listing 4).

Listing 4: Shutting down the sequence grabber

QTCap_Stop

void QTCap_Stop (void)
{
   if (gSeqGrabber != NULL) {
      SGStop(gSeqGrabber);
      CloseComponent(gSeqGrabber);
      gSeqGrabber = NULL;
   }

   if (gMonitor != NULL) {
      DisposeDialog(gMonitor);
      gMonitor = NULL;
   }
}

You’ll notice that we didn’t explicitly close the sequence grabber channel component instances gVideoChannel or gSoundChannel. The sequence grabber does that automatically for us when we call CloseComponent on the sequence grabber component instance we opened.

Channel Settings

QTCapture includes menu items that display dialog boxes in which the user can configure the settings of the video and sound channels. We’ve already seen the sound settings dialog box (in Figures 4 and 5) and the video settings dialog box (in Figure 3); Figure 6 shows another pane of the video settings dialog box.

Figure 6: The video settings dialog box

In both cases, we display the settings dialog box by calling the SGSettingsDialog function, passing in our instances of the sequence grabber component and the appropriate channel component. There are, however, a few extra details that we need to consider when we call SGSettingsDialog.

Handling Update Events

As you can see, the settings dialog boxes are movable modal dialog boxes. This means that, on Macintosh computers, we’ll also need to specify a modal dialog filter function to handle idle events and to pass update events to windows located behind the settings dialog box. If we didn’t do this, and if the user were to move the settings dialog box on top of another QTCapture window and then move it away, that window would not get redrawn. (On Windows computers, as we’ve seen in earlier articles, paint messages are sent directly to the affected window, so we don’t need a modal dialog filter function.)

We’ve already developed a basic modal dialog filter function that is able to redraw any movie or image windows that our application has open. (See, for instance, “Honey, I Shrunk the Kids” in MacTech, February 2001.) In the present case, we also want to redraw the monitor window itself, in case it gets covered up and then uncovered by a settings dialog box. The sequence grabber provides the SGUpdate function, which instructs the sequence grabber to refresh its display. In theory, we could use SGUpdate here, except for one small problem: QuickTime steals our sequence grabber component instance while the video settings dialog is displayed. Look again at Figure 6 and notice that the right-hand side of the dialog contains a pane in which the previewed video data is displayed. The input for that pane is provided by our very own sequence grabber component, gSeqGrabber. So we can call SGUpdate until the cows come home and our monitor window will never get refreshed.

There is a simple workaround to this theft. We can take a snapshot of the monitor window just before we call SGSettingsDialog to display a settings dialog box, and then redraw the monitor window using that snapshot whenever necessary. Listing 5 shows our function QTCap_GetChannelSettings, which we’ll call to display a video or sound settings dialog box. As you can see, we call SGGrabPict to get a picture that contains the current image in the monitor window. Then we call SGSettingsDialog and later clean up by disposing of the grabbed picture.

Listing 5: Displaying a settings dialog box

QTCap_GetChannelSettings

static ComponentResult QTCap_GetChannelSettings 
            (SGChannel theChannel)
{
   SGModalFilterUPP      myFilterUPP = NULL;
   ComponentResult         myErr = noErr;

   // get rid of any existing monitor picture
   if (gMonitorPICT != NULL) {
      KillPicture(gMonitorPICT);
      gMonitorPICT = NULL;
   }

   // get the picture currently in the monitor window
   SGGrabPict(gSeqGrabber, &gMonitorPICT, NULL, 0, 
            grabPictOffScreen);

   // display the settings dialog box
#if TARGET_OS_MAC
   myFilterUPP = NewSGModalFilterUPP(QTCap_SGModalFilterProc);
#endif

   myErr = SGSettingsDialog(gSeqGrabber, theChannel, 0, NULL, 
            0L, myFilterUPP, (long)gMonitor);

#if TARGET_OS_MAC
   DisposeSGModalFilterUPP(myFilterUPP);
#endif

   // get rid of the monitor picture
   if (gMonitorPICT != NULL) {
      KillPicture(gMonitorPICT);
      gMonitorPICT = NULL;
   }

   return(myErr);
}

Strictly speaking, we need to call SGGrabPict only when we’re about to display the video settings dialog box. But the code for redrawing the uncovered monitor window is in fact much simpler if we grab the picture in the monitor window in both cases (that is, for both the video and sound settings dialog boxes).

Listing 6 shows our complete sequence grabber modal dialog filter function. It’s pretty much identical to the modal dialog filter functions we’ve encountered hitherto, except that it contains code to determine whether the window to be updated is the monitor window gMonitor. If it is, we get the size of the monitor window and draw the saved snapshot into that window by calling DrawPicture. As you can see, QTCap_SGModalFilterProc assumes that the theRefCon parameter is a pointer to the monitor dialog box. If you look back at Listing 5, you’ll see that indeed we pass gMonitor as the last parameter to SGSettingsDialog.

Listing 6: Handling events

QTCap_SGModalFilterProc

#if TARGET_OS_MAC
static PASCAL_RTN Boolean QTCap_SGModalFilterProc 
            (DialogPtr theDialog, const EventRecord *theEvent, 
            short *theItemHit, long theRefCon)
{
   Boolean            myEventHandled = false;
   WindowPtr         myWindow = NULL;
   RgnHandle         myWindowRgn = NULL;
   GrafPtr            mySavedPort;
   Rect                  myRect;
   DialogPtr         myMonitor = (DialogPtr)theRefCon;

   switch (theEvent->what) {
      case updateEvt:
         // find out which window needs to be updated
         myWindow = (WindowPtr)theEvent->message;
         if (myWindow == GetDialogWindow(myMonitor)) {
            // update the monitor window, using the stored picture
            GetPort(&mySavedPort);
            MacSetPort(GetWindowPort(myWindow));

#if TARGET_API_MAC_CARBON
            GetPortBounds(GetDialogPort(myMonitor), &myRect);
#else
            myRect = myWindow->portRect;
#endif

            // draw the saved monitor picture into the monitor window
            if (gMonitorPICT != NULL)
               DrawPicture(gMonitorPICT, &myRect);

            // clear the update region
            BeginUpdate(myWindow);
            EndUpdate(myWindow);

            MacSetPort(mySavedPort);
            myEventHandled = true;
         } else if ((myWindow != NULL) && 
                  (myWindow != GetDialogWindow(theDialog))) {
            // update the specified window, if it’s behind the modal dialog box
            QTFrame_HandleEvent((EventRecord *)theEvent);
            myEventHandled = false;
         }
         break;

      case nullEvent:
         // do idle-time processing for all open windows in our window list
         if (gAppInForeground) 
            QTFrame_IdleMovieWindows();

         myEventHandled = false;
         break;

      default:
         myEventHandled = false;
         break;
   }

   // let the OS’s standard filter proc handle the event, if it hasn’t already been handled
   if (gHasNewDialogCalls && (myEventHandled == false))
      myEventHandled = StdFilterProc(theDialog, 
            (EventRecord *)theEvent, theItemHit);

   return(myEventHandled);
}
#endif

Displaying the Settings Dialog Boxes

Now we’ve got the necessary tools we need to display the sound and video settings dialog boxes. Listing 7 shows the definition of the QTCap_GetSoundSettings function. Pretty simple, eh?

Listing 7: Displaying the sound settings dialog box

QTCap_GetSoundSettings

void QTCap_GetSoundSettings (void)
{
   QTCap_GetChannelSettings(gSoundChannel);
}

For the video settings dialog box, however, we need to do a little more work. The principal complication is that some of the user’s selections may cause the video digitizer’s active source rectangle to change. Indeed, the user can even change the video digitizer itself, by selecting a new video input source. So we need to pay attention to any size changes that may occur and then recalculate and reset the size of the monitor window accordingly.

Before we call QTCap_GetChannelSettings on the video channel, we want to pause the preview operation. We can do that by calling SGPause:

SGPause(gSeqGrabber, true);

At this point, we can go ahead and call QTCap_GetChannelSettings, passing in the video channel component instance:

myErr = QTCap_GetChannelSettings(gVideoChannel);
if (myErr != noErr)
   goto bail;

If QTCap_GetChannelSettings returns successfully, we’ll retrieve the new video boundary rectangle:

SGGetSrcVideoBounds(gVideoChannel, &myNewActiveVideoRect);

The video source boundary rectangle defines the size of the source video image being captured by the video channel. The active source rectangle is usually a part of the video source boundary rectangle.

Now we need to adjust the size of the monitor window if the active source rectangle has changed size (that is, if myNewActiveVideoRect differs from gActiveVideoRect). Listing 8 shows the code we execute in that case.

Listing 8: Adjusting the size of the monitor window

QTCap_GetVideoSettings

if (!MacEqualRect(&gActiveVideoRect, &myNewActiveVideoRect)) 
{
   short         myDivisor = 1;      // assume gFullSize

   if (gQuarterSize)
      myDivisor = 4;
   else if (gHalfSize)
      myDivisor = 2;

   myWidth = (myNewActiveVideoRect.right – 
                     myNewActiveVideoRect.left) / myDivisor;
   myHeight = (myNewActiveVideoRect.bottom – 
                     myNewActiveVideoRect.top) / myDivisor;

   gActiveVideoRect = myNewActiveVideoRect;
   SizeWindow(GetDialogWindow(gMonitor), myWidth, myHeight, 
            false);

   GetPortBounds(GetDialogPort(gMonitor), &myRect);
   SGSetChannelBounds(gVideoChannel, &myRect);
}

Note that we resize the monitor window and then reset the display boundary rectangle. Listing 9 shows our complete definition of QTCap_GetVideoSettings.

Listing 9: Displaying the video settings dialog box

QTCap_GetVideoSettings

void QTCap_GetVideoSettings (void)
{
   Rect                        myNewActiveVideoRect;
   short                       myWidth, myHeight;
   GrafPtr                     mySavedPort;
   SGModalFilterUPP            myFilterUPP = NULL;
   Rect                        myRect;
   ComponentResult             myErr = noErr;

   // get our current state
   GetPort(&mySavedPort);

   // pause previewing
   SGPause(gSeqGrabber, true);

   // display the video settings dialog box
   myErr = QTCap_GetChannelSettings(gVideoChannel);
   if (myErr != noErr)
      goto bail;

   // retrieve the user’s choices
   SGGetSrcVideoBounds(gVideoChannel, &myNewActiveVideoRect);

   // set up our port
   SetPortDialogPort(gMonitor);

   // has our active rectangle changed?
   // if so, it’s because our video standard changed (e.g., NTSC to PAL) 
   // and we need to adjust our monitor window
   if (!MacEqualRect(&gActiveVideoRect, 
            &myNewActiveVideoRect)) {
      short         myDivisor = 1;      // assume gFullSize

      if (gQuarterSize)
         myDivisor = 4;
      else if (gHalfSize)
         myDivisor = 2;

      myWidth = (myNewActiveVideoRect.right – 
                        myNewActiveVideoRect.left) / myDivisor;
      myHeight = (myNewActiveVideoRect.bottom – 
                        myNewActiveVideoRect.top) / myDivisor;

      gActiveVideoRect = myNewActiveVideoRect;
      SizeWindow(GetDialogWindow(gMonitor), myWidth, myHeight, 
            false);

      GetPortBounds(GetDialogPort(gMonitor), &myRect);
      SGSetChannelBounds(gVideoChannel, &myRect);
   }

bail:
   MacSetPort(mySavedPort);

#if !TARGET_OS_MAC
   // this is necessary, for now, to get the grab to start again after the dialog goes away;
   // for some reason the video destRect never gets reset to point back to the monitor 
   // window
   SGSetChannelBounds(gVideoChannel, &(gMonitor->portRect));
#endif

   // restart previewing
   SGPause(gSeqGrabber, false);
}

As you can see, on Windows we call SGSetChannelBounds to reset the channel bounds rectangle to the size of the monitor window.

Monitor Window Size

While we’re on the subject of resizing the monitor window to fit the active source rectangle, let’s see how QTCapture handles the three menu items that adjust the size of the monitor window. The application function QTApp_HandleMenu contains the lines of code shown in Listing 10.

Listing 10: Handling the size-related menu items

QTCap_GetVideoSettings

case IDM_QUARTER_SIZE:
   QTCap_ResizeMonitorWindow(4);
   myIsHandled = true;
   break;

case IDM_HALF_SIZE:
   QTCap_ResizeMonitorWindow(2);
   myIsHandled = true;
   break;

case IDM_FULL_SIZE:
   QTCap_ResizeMonitorWindow(1);
   myIsHandled = true;
   break;

In all three cases, we call the function QTCap_ResizeMonitorWindow, passing in the appropriate divisor. QTCap_ResizeMonitorWindow is defined in Listing 11; it should be fairly clear, given the similar code we just encountered in Listing 9.

Listing 11: Resizing the monitor window

QTCap_ResizeMonitorWindow

void QTCap_ResizeMonitorWindow (short theDivisor)
{
   Rect                        myRect;
   short                     myWidth, myHeight;
   GrafPtr                  mySavedPort;
   ComponentResult         myErr = noErr;

   // calculate the new width and height
   myWidth = (gActiveVideoRect.right - gActiveVideoRect.left) 
            / theDivisor;
   myHeight = (gActiveVideoRect.bottom - gActiveVideoRect.top) 
            / theDivisor;

   gQuarterSize = (theDivisor == 4);
   gHalfSize = (theDivisor == 2);
   gFullSize = (theDivisor == 1);

   // resize the monitor window
   GetPort(&mySavedPort);
   SetPortDialogPort(gMonitor);
   
   SGPause(gSeqGrabber, true);

   SizeWindow(GetDialogWindow(gMonitor), myWidth, myHeight, 
            false);

   GetPortBounds(GetDialogPort(gMonitor), &myRect);
   SGSetChannelBounds(gVideoChannel, &myRect);

   MacSetPort(mySavedPort);
   SGPause(gSeqGrabber, false);
}

We resize the monitor window to its new height and width by calling SizeWindow. We also reset the video channel’s display boundary rectangle (the rectangle in which the previewed data is displayed) by calling SGSetChannelBounds. Because the display boundary rectangle completely fills the content area of the monitor window, we are able to keep the geometry calculations fairly simple. In a more typical case, where the preview occupies only part of a window, we’d need to do some more complicated calculations. If you are interested, take a look at the HackTV sample code package cited at the end of this article.

Recording

Finally it’s time to see how to use the sequence grabber to record captured video and sound data into a movie file. This is actually a fairly simple task, as the sequence grabber provides the SGStartRecord and SGStop functions that we can use to start and stop recording. First, however, we need to tell the sequence grabber where to put the captured data.

Setting the Output File

The first thing we want to do is have the user select a file to hold the captured data. We’ll use our framework function QTFrame_PutFile and then call DeleteMovieFile if the selected file already exists and the user tells us to overwrite that existing file, as shown in Listing 12.

Listing 12: Eliciting an output file from the user

QTCap_Record

QTFrame_PutFile(myPrompt, myFileName, &myFile, 
            &myIsSelected, &myIsReplacing);
myErr = myIsSelected ? noErr : userCanceledErr;
if (myErr != noErr)
   goto bail;

// delete any existing the movie file, if the user so instructs
if (myIsReplacing)
   DeleteMovieFile(&myFile);

Next we’ll call the SGSetDataOutput function to set the selected file as the output file for the recorded data. The sequence grabber stores the data in the file as a QuickTime movie, complete with the requisite movie metadata (that is, the movie atom) and the appropriate sound and video tracks.

myErr = SGSetDataOutput(gSeqGrabber, &myFile, seqGrabToDisk);

The third parameter to SGSetDataOutput is a set of flags that control various aspects of the recording operation. Currently these flags are available:

enum {
   seqGrabToDisk                             = 1,
   seqGrabToMemory                           = 2,
   seqGrabDontUseTempMemory                  = 4,
   seqGrabAppendToFile                       = 8,
   seqGrabDontAddMovieResource               = 16,
   seqGrabDontMakeMovie                      = 32,
   seqGrabPreExtendFile                      = 64,
   seqGrabDataProcIsInterruptSafe            = 128,
   seqGrabDataProcDoesOverlappingReads       = 256
};

The first two flags, seqGrabToDisk and seqGrabToMemory, are mutually exclusive. The seqGrabToDisk flag tells the sequence grabber to write the captured data to the output file as the data is captured; the seqGrabToMemory flag tells the sequence grabber first to record the data into memory and then to write it into the output file only when the recording operation is complete. Using the seqGrabToMemory flag can result in better performance (that is, fewer dropped frames) but it limits the amount of recorded data to the memory available to our application. (It’s worth noting that this technique for avoiding dropped frames is far less necessary these days, as hard disks are significantly faster than in the early days of QuickTime.) As you can see above, QTCapture specifies the seqGrabToDisk flag. The remaining flags are for more specialized capture operations and we won’t consider them further.

Setting Channel Output Files

By default, each open channel writes its data into the file specified by the SGSetDataOutput function. It’s possible, however, to configure the sequence grabber to record different channels into different files. To do this, we need to create a new sequence grabber output and attach that output to a particular channel. We create a new channel output by calling the SGNewOutput function, and we attach that output to a channel by calling the SGSetChannelOutput function.

When we call SGNewOutput, we need to specify the output file by passing in a data reference to the destination file; this is unlike SGSetDataOutput, where we passed a pointer to a file specification record. (See “Somewhere I’ll Find You” in MacTech, October 2000 for a discussion of working with data references.) In QTCapture, we prompt the user for a channel output file by calling QTFrame_PutFile, so we need to create an alias data reference for that file:

myErr = QTNewAlias(&myFile, &myAliasHandle, true);

And then we can call SGNewOutput and SGSetChannelOutput, like this:

SGNewOutput(gSeqGrabber, (Handle)myAliasHandle, rAliasType, 
            seqGrabToDisk, &myOutput);
SGSetChannelOutput(gSeqGrabber, theChannel, myOutput);

Notice that SGNewOutput also takes a parameter that specifies the desired recording options; in this case, we’ll pass the same flag, seqGrabToDisk, that we earlier passed to SGSetDataOutput.

QTCapture maintains a global variable, gSplitTracks, that indicates whether the user wants to capture video and sound data into different files. Before we begin recording, we inspect that variable and, if necessary, prompt the user to select the channel output files. Listing 13 shows the code that does this.

Listing 13: Eliciting channel output files from the user

QTCap_Record

if ((gSoundChannel != NULL) && gRecordSound && 
   (gVideoChannel != NULL) && gRecordVideo && gSplitTracks) {
   myErr = QTCap_SetTrackFile(gVideoChannel, kVideoSavePrompt, 
            kVideoSaveMovieFileName);
   if (myErr != noErr)
      goto bail;

   myErr = QTCap_SetTrackFile(gSoundChannel, kSoundSavePrompt, 
            kSoundSaveMovieFileName);
   if (myErr != noErr)
      goto bail;
}

As you can see, we call the function QTCap_SetTrackFile, defined in Listing 14, to do most of the work. QTCap_SetTrackFile just assembles the pieces we’ve encountered so far in this section.

Listing 14: Setting a channel output file

QTCap_SetTrackFile

static ComponentResult QTCap_SetTrackFile 
            (SGChannel theChannel, char *thePrompt, 
               char *theDefaultName)
{
   FSSpec                  myFile;
   Boolean               myIsSelected = false;
   Boolean               myIsReplacing = false;
   StringPtr             myPrompt = 
            QTUtils_ConvertCToPascalString(thePrompt);
   StringPtr             myFileName = 
            QTUtils_ConvertCToPascalString(theDefaultName);
   SGOutput               myOutput;
   AliasHandle         myAliasHandle = NULL;
   OSErr                  myErr = noErr;

   // prompt the user for new file name
   QTFrame_PutFile(myPrompt, myFileName, &myFile, 
            &myIsSelected, &myIsReplacing);
   myErr = myIsSelected ? noErr : userCanceledErr;
   if (myErr != noErr)
      goto bail;

   myErr = QTNewAlias(&myFile, &myAliasHandle, true);
   if (myErr != noErr)
      goto bail;

   // create an output from this file
   myErr = SGNewOutput(gSeqGrabber, (Handle)myAliasHandle, 
            rAliasType, seqGrabToDisk, &myOutput);
   if (myErr != noErr)
      goto bail;

   // associate this output with the specified channel
   myErr = SGSetChannelOutput(gSeqGrabber, theChannel, 
            myOutput);

bail:
   free(myPrompt);
   free(myFileName);

   if (myAliasHandle != NULL)
      DisposeHandle((Handle)myAliasHandle);

   return(myErr);
}

Keep in mind that we now have three files floating around. We have the main output file (set by a call to SGSetDataOutput); this file contains the movie atom, which in turn contains two track atoms. These track atoms contain references to the two channel output files (set by calls to SGSetChannelOutput). The channel output files are media files; they cannot be opened by QuickTime-savvy applications directly. Instead, they are opened only indirectly, whenever the main output file is opened.

Recording the Captured Data

So we’ve set the main output file and, if desired, the channel output files. It’s time to start recording some captured data into those files. As indicated earlier, we do this by calling SGStartRecord:

myErr = SGStartRecord(gSeqGrabber);

Once we’ve successfully called SGStartRecord, the sequence grabber will capture sound and video data into the specified output file or files until we tell it to stop (by calling SGStop). We need to give some processor time to the sequence grabber, just as we did during previewing, by calling SGIdle. In QTCapture, we’re going to use a fairly cheesy strategy of just recording until the user clicks the mouse button:

while (!Button() && (myErr == noErr))
   myErr = SGIdle(gSeqGrabber);

SGStop(gSeqGrabber);

This strategy has the benefit of simplicity and also of providing maximum processing time to the sequence grabber, but it’s certainly not appropriate for a real-life capture application. The tasks of providing a better user-interface for starting and stopping the recording process and of rewriting the code to use the call to SGIdle in our idle-event handler QTApp_Idle are, as you’ve probably guessed, left as exercises to the reader. (Calling Button in a while loop is particularly obnoxious when running on Mac OS X; see the SGDataProcSample package cited at the end of this article for some sample code that uses Carbon events to call SGIdle.)

Listing 15 shows our complete recording function, QTCap_Record.

Listing 15: Recording captured data into a file

QTCap_Record

void QTCap_Record (void)
{
   FSSpec                  myFile;
   Boolean               myIsSelected = false;
   Boolean               myIsReplacing = false;
   StringPtr             myPrompt = 
         QTUtils_ConvertCToPascalString(kCapSavePrompt);
   StringPtr             myFileName = 
         QTUtils_ConvertCToPascalString(kCapSaveMovieFileName);
   long                     myFlags = createMovieFileDontOpenFile | 
                                 createMovieFileDontCreateMovie | 
                                 createMovieFileDontCreateResFile;
   ComponentResult      myErr = noErr;

   // stop everything while the dialogs are up
   SGStop(gSeqGrabber);

   // prompt the user for new file name
   QTFrame_PutFile(myPrompt, myFileName, &myFile, 
            &myIsSelected, &myIsReplacing);
   myErr = myIsSelected ? noErr : userCanceledErr;
   if (myErr != noErr)
      goto bail;

   // delete any existing the movie file, if the user so instructs
   if (myIsReplacing)
      DeleteMovieFile(&myFile);

   myErr = SGSetDataOutput(gSeqGrabber, &myFile, 
            seqGrabToDisk);
   if (myErr != noErr)
      goto bail;

   // ask for separate video and sound track files, if requested
   if ((gSoundChannel != NULL) && gRecordSound && 
         (gVideoChannel != NULL) && gRecordVideo && 
            gSplitTracks) {
      myErr = QTCap_SetTrackFile(gVideoChannel, 
            kVideoSavePrompt, kVideoSaveMovieFileName);
      if (myErr != noErr)
         goto bail;

      myErr = QTCap_SetTrackFile(gSoundChannel, 
            kSoundSavePrompt, kSoundSaveMovieFileName);
      if (myErr != noErr)
         goto bail;
   }

   // if not recording sound or video, then disable those channels
   if ((gSoundChannel != NULL) && !gRecordSound)
      SGSetChannelUsage(gSoundChannel, 0);

   if ((gVideoChannel != NULL) && !gRecordVideo)
      SGSetChannelUsage(gVideoChannel, 0);

   // attempt to recover the preview area obscured by dialogs
#if TARGET_OS_WIN32
   UpdatePort(gMonitor);
#endif
   SGUpdate(gSeqGrabber, 0);

   // create a movie file for the destination movie
   myErr = CreateMovieFile(&myFile, sigMoviePlayer, 
            smSystemScript, myFlags, NULL, NULL);
   if (myErr != noErr)
      goto bail;

   FlushEvents(mDownMask + mUpMask, 0);

   // record until the user clicks the mouse button
   myErr = SGStartRecord(gSeqGrabber);
   if (myErr != noErr)
      goto bail;

   while (!Button() && (myErr == noErr))
      myErr = SGIdle(gSeqGrabber);

   // if we recorded until we ran out of space, then allow SGStop to be called to write the 
   // movie resource; the assumption here is that the data output filled up but the disk has 
   // enough free space left to write the movie resource
   if (!((myErr == dskFulErr) || (myErr != eofErr)))
      goto bail;

   // stop the recording that’s currently happening
   myErr = SGStop(gSeqGrabber);
   SGStartPreview(gSeqGrabber);

bail:
   free(myPrompt);
   free(myFileName);

   if (myErr == noErr)
      return;

   SGPause(gSeqGrabber, false);
   SGStartPreview(gSeqGrabber);
}

Notice that once we’re done recording, we restart the previewing process by calling SGStartPreview.

Conclusion

We now know how to use the sequence grabber and sequence grabber channel components to preview and capture video and sound data from a camera or other audiovisual device attached to our computer. We’ve seen how to display the settings dialog boxes that permit the user to configure the various channels, and we’ve also seen how to set the sound and video channels to capture into different output files. And we did all of this without knowing very much at all about the various components that do all the low-level work. That’s part of the beauty of the sequence grabber: it gives us a simple, high-level interface to a set of fairly complex operations.

Saving this captured data in a file is great stuff, of course, but there are certainly other things we might want to do with it. For instance, we might want to send it out it over a network, so that people located remotely can watch and listen to our data. In the next article, we’ll see how QuickTime can help us do that.

Credits

Thanks are due to Kevin Marks, who reviewed this article and provided a number of helpful comments. The code in QTCapture is based heavily on an existing sample code package called HackTV. It and several other useful sample code packages related to capturing (including the SGDataProcSample package mentioned earlier) are available on-line at
http://developer.apple.com/samplecode/Sample_Code/QuickTime/Capturing.htm.

Tim Monroe is a member of the QuickTime engineering team. You can contact him at monroe@apple.com.