TweetFollow Us on Twitter

Programming With QuickTimeVR

Volume Number: 13 (1997)
Issue Number: 7
Column Tag: Multimedia

Programming With QuickTime VR

by Tim Monroe and Bryce Wolfson, Apple Computer, Inc.

A look at the new API for managing QuickTime VR movies

Introduction

QuickTime VR is the part of the QuickTime Media Layer that allows users to interactively explore and examine photorealistic, three-dimensional virtual worlds and objects. For several years, MacOS and Windows users have been able to play back QuickTime VR content using standard navigation controls. With version 2.0, QuickTime VR now supports a C API that allows developers to customize and extend the user's virtual experience. Here you'll find everything you need to know to get started supporting QuickTime VR in your application.

QuickTime VR is certainly one of the hottest Apple technologies today. Dozens and dozens of CD titles have appeared in the past months that depend on QuickTime VR. More significantly, the World Wide Web now serves up literally thousands of QuickTime VR movies. A large part of this popularity stems from the fact that the amount of data required to display a complex, photorealistic panorama using QuickTime VR is much smaller than would be required to model that panorama in detail using a standard 3D graphics application. This, together with the fact that no run-time rendering is occurring, drastically reduces the amount of CPU horsepower and RAM required to immerse the user in a realistic 3D environment. QuickTime VR can transport the user from the tightest passageway deep inside the pyramids to the wide-open expanse of the space shuttle payload bay.

But, to paraphrase Al Jolson, you ain't seen nothin' yet, folks. Apple has recently introduced a C language programming interface to QuickTime VR -- called the QuickTime VR Manager -- that provides a large set of tools for extending and customizing the user's virtual experience. The QuickTime VR Manager provides the necessary hooks for you to add your own custom processing to VR movie playback and to integrate QuickTime VR content playback with other technologies, particularly with other multimedia technologies. For instance, you can very easily integrate Apple's SoundSprocket with QuickTime VR to attach sounds to specific locations in a panorama. Or, you can play QuickTime movies on the screen of a television in a panorama. Or, you can embed QuickDraw 3D objects (even moving objects!) in a panorama or object node. Take your favorite technology and, chances are, there's a cool way to integrate it with a VR panorama or object.

In this article, we'll explain some basic ways of using the QuickTime VR Manager in your application. We start by reviewing some simple but important aspects of QuickTime movie support. Then we move into programmatic control of QuickTime VR movies by showing how to use standard Movie Toolbox and movie controller functions to operate on QuickTime VR movies. Finally, we'll describe the QuickTime VR Manager itself and present some code that demonstrates a few of its capabilities.

Before reading this article, you should be familiar with the basic capabilities and operations of QuickTime VR. At the very least, you should have opened some panoramas or object movies using a QuickTime player (for instance, MoviePlayer or QTVRPlayer) and have a good understanding of the VR user interface (panning and tilting, zooming in and out, triggering hot spots, and so forth). You can also get the necessary background information by looking at the first dozen pages of the book Virtual Reality Programming With QuickTime VR 2.0, the developer documentation for the QuickTime VR Manager.

Back To Basics

The first thing to keep in mind is that QuickTime VR movies are just QuickTime movies, at least in the way the movie data is stored in files. The image data for panoramas and objects is stored in standard QuickTime video tracks in movie files. Other data (such as names of the nodes in a scene) is stored in data atoms in the movie resources. What's different about QuickTime VR is the way in which the image data is handled. In a QuickTime movie, frames of a movie are read from a video track and displayed in sequence to the user. In a QuickTime VR movie, the image handling is more complicated, because the image to be displayed depends more heavily on user interaction. In an object movie, for instance, the user can pan left or right, tilt up or down, or zoom in and out (among other things). These actions typically require a new frame from the movie's video track to be displayed. The new frame, however, may or may not follow the current frame in the movie's video track.

Displaying the appropriate panorama or object images in response to user actions is the responsibility of the QuickTime VR movie controller, which is a standard QuickTime movie controller. Every QuickTime VR movie contains a special piece of user data that identifies the movie controller for the movie. This user data is examined by the Movie Toolbox when an application calls the NewMovieController function to create a movie controller for the movie. If the QuickTime VR extension is installed and the QuickTime VR movie controller component is therefore available, the Movie Toolbox locates that controller and assigns it to that movie.

Now here's the payoff: if you've done things right, your QuickTime application already supports QuickTime VR! Go ahead and try it: launch your QuickTime-savvy application, select Open from the File menu, and then choose a QuickTime VR movie. (Easier still, just drag the VR movie icon onto your application's icon.) If everything goes according to plan, the VR movie will open correctly and you'll be able to navigate within the panorama or manipulate the object in all the right ways. This is because, as we've just said, the QuickTime VR file contains information that specifies the QuickTime VR movie controller instead of a QuickTime movie controller. It couldn't be any easier.

This payoff, however, has a catch: to get VR movie playback, you have to support QuickTime in the right way. Part of what that means is that you have to call NewMovieController to associate a movie controller with a movie. Years ago, Peter Hoddie voiced this warning in a develop column: "When you need a user interface for playing a movie, you should use NewMovieController to create a movie controller appropriate for use with that movie. A common mistake is to instead use the Component Manager routine FindNextComponent or OpenDefaultComponent to locate a movie controller. This finds the first movie controller in the system's list of registered components. QuickTime has always contained only one movie controller, so this worked fine. However, future versions of QuickTime will almost certainly include other movie controllers, so the first one isn't necessarily the most appropriate one." (Hoddie, p. 22)

These "future versions" of QuickTime are here now, and one of them is QuickTime VR. So, to support VR movies, you must call NewMovieController instead of the Component Manager. More generally, you should make sure that you've followed all the advice in Hoddie's important article. It won't take you long to do so, and it will make it easier to support QuickTime in all its manifestations.

Beyond the Basics

Once you've implemented basic -- and correct -- QuickTime movie playback support in your application, there are still a few other things you should do to handle QuickTime VR movies. In particular, you'll want to enable and disable menu items correctly and to display the correct cursor when it's outside the movie window. Happily, you can do each of these things with just a few lines of code. We're supposing throughout that you want your application to handle both QuickTime and QuickTime VR movies; if instead you wanted to build just a QuickTime VR player, some of these steps (for instance, worrying about the Select All menu command) wouldn't be necessary.

The first thing you'll want to do is support the standard movie editing commands for QuickTime movies but also disable any Edit menu items that don't make sense for QuickTime VR movies. When you first open a QuickTime or QuickTime VR movie, you can call MCEnableEditing to turn on editing (which by default is off):

MCEnableEditing(myMC, true);      // enable movie controller editing

Here myMC is the movie controller. We'll keep track of this and other information by setting fields in a window object record, a data structure associated with each window that contains a QuickTime or QuickTime VR movie; see later on in this article for the exact structure of a window object record. As recommended in Hoddie's article, we use the MCSetUpEditMenu function in our menu-adjusting routine to enable and disable items in the Edit menu:

myMC = (**myWindowObject).fController;
if (myMC != NULL)
   MCSetUpEditMenu(myMC, 0L, GetMHandle(mEdit));

The QuickTime VR movie controller disables most of the Edit menu items, since they don't apply to VR movies. It does not however disable the Select All item, so you must do that yourself, like this:

myErr = MCGetControllerInfo(myMC, &myControllerFlags);
if (myErr != noErr)
   myControllerFlags = 0L;
isEditingEnabled = 
   (mcInfoEditingEnabled & myControllerFlags) != 0;
if (isEditingEnabled)
   EnableItem(GetMHandle(mEdit), iSelectAll);
else
   DisableItem(GetMHandle(mEdit), iSelectAll);

Next, the QuickTime VR movie controller manages the cursor whenever it's within a VR movie, changing its shape at appropriate times (such as when the cursor is over a hot spot). The QuickTime VR movie controller doesn't restore the cursor to the arrow shape when the cursor moves outside of the movie. To do that, you can put this code in your idle event processing code:

if (theWindow == FrontWindow()) {
   Rect   myRect;
   GetMouse(&myPoint);
   MCGetControllerBoundsRect(myMC, &myRect);
   if (!PtInRect(myPoint, myRect))
      InitCursor();
}

This cursor-adjusting code is currently needed only for QuickTime VR movies, but should probably be called by any QuickTime-savvy application.

You'll notice that we've used several movie controller component functions with a QuickTime VR movie. This underscores once again the fact that QuickTime VR movies are just special kinds of QuickTime movies. For instance, you can hide the controller bar by executing this code:

MCSetVisible(myMC, false);

In general, you can use any movie controller functions with QuickTime VR movies, except those that specifically relate to time or movie editing. It would make no sense, for example, to issue the movie controller action mcActionGoToTime on a QuickTime VR movie. When QuickTime VR movies do involve temporal aspects (for instance, an animated object movie has a play rate), the QuickTime VR Manager provides functions that you can use to manipulate those aspects.

The QuickTime VR movie controller also supports movie controller action filters. That is, you can intercept a VR movie's controller actions and react accordingly. So virtually the entire QuickTime programming interfaces are applicable also to QuickTime VR movies.

So far, we've shown how to support QuickTime VR movie playback, and even provide a significant level of customization of the movie playback and user interface, just using standard QuickTime movie controller functions. The sample application VRShell provides a concrete example of this capability. It's built on MovieShell, a QuickTime-savvy framework written by Apple DTS. All of our remaining sample applications are built in turn on VRShell.

Overview Of The QuickTime VR Manager

The QuickTime VR Manager provides a large number of capabilities that you can use to customize and extend the user's virtual experience of panoramas and objects. Here we'll summarize the basic capabilities of the QuickTime VR Manager. Then, in the following sections, we'll illustrate how to use some of them. In this article, we'll keep things fairly simple; in the future, we hope to illustrate some of the more advanced capabilities of the QuickTime VR Manager.

The QuickTime VR Manager provides these main capabilities:

  • Positioning. A VR movie file contains a scene, which is a collection of one or more nodes. Each node is either a panoramic node (a "panorama") or an object node (an "object") and is uniquely identified by its node ID. Within a panoramic node, the user's view is determined by three factors: the pan angle, the tilt angle, and the vertical field of view. For objects, the view is also determined by the view center. The QuickTime VR Manager provides functions to get and set any of these items. For instance, you can programmatically spin an object around by repeatedly incrementing the current pan angle.
  • Hot spot handling. You can use the QuickTime VR Manager to manage any hot spots in a panorama or object. For instance, you can trigger a hot spot programmatically (that is, simulate a click on the hot spot), enable and disable hot spots, determine whether the cursor is over a hot spot, find all visible hot spots, and so forth. You can also install a callback routine that is called whenever the cursor is over an enabled hot spot.
  • Custom node entering and leaving behaviors. The QuickTime VR Manager allows you to perform actions whenever the user enters a new node or leaves the current node. For instance, you might play a custom sound when the user enters a particular node. Current versions of QuickTime VR support scenes that contain both panoramic and object nodes. If you want to treat object nodes differently from panoramic nodes, you can use node-entering and node-leaving procedures to do any necessary processing.
  • Getting information. You can use the QuickTime VR Manager to get information about a scene or about a specific node. For instance, you might want to determine the ID and type of the current node. Much of the information about scenes and nodes is stored in atoms in the movie file. To get information about a scene or node that isn't provided directly by the QuickTime VR Manager, you'll have to use the QuickTime atom container functions (introduced in QuickTime version 2.1) to extract information from those atoms.
  • Intercepting QuickTime VR Manager functions. You can intercept calls to some QuickTime VR Manager functions to augment or modify their behavior. For example, to assign behaviors to custom hot spots, you can install an intercept routine to be called whenever a hot spot is triggered. Your intercept routine would check the type of the triggered hot spot and then perform the correct actions for that type. Another common use of intercept routines is to intercept positioning functions (changing the pan, tilt, and field of view) and adjust environmental factors accordingly.
  • Accessing offscreen panorama buffers. QuickTime VR maintains two offscreen buffers for each panorama, a back buffer and a prescreen buffer. The back buffer contains some or all of the data in a panorama image track, which is a standard QuickTime video track whose frames contain slices of a cylindrically warped and rotated version of the original panorama. The prescreen buffer contains the unwarped and unrotated image that is about to be copied to the screen.

You can use QuickTime VR Manager functions to access the back buffer and the prescreen buffer. Which buffer you draw into is determined by the effect you want to achieve. To overlay a graphic (such as a head's-up display) that is unaffected by the user's panning, tilting, or zooming, you would draw into the prescreen buffer. To overlay a graphic that is affected by changes in the view, you would draw into the back buffer. Back buffer drawing is a bit tricky, however, because the images you draw there must be rotated and warped if they are to appear correctly on the screen. (Note: in future versions of QuickTime VR, the back buffer might not be rotated. The QuickTime VR Manager provides a way to check whether it is rotated or not.)

This list is not exhaustive. The QuickTime VR Manager provides many other capabilities as well. For a complete description, see the book Virtual Reality Programming With QuickTime VR 2.0.

Starting Up

Before you can do these neat things with the QuickTime VR Manager, you must do a little setting up (over and beyond what's required for using QuickTime). First, of course, you must ensure that the QuickTime VR Manager is available in the current operating environment. As you'd expect, there are several Gestalt selectors that you can use to see whether the QuickTime VR Manager is available and what features it has. Here we'll spare you the details on calling Gestalt; consult the sample code (or the reference book) if you absolutely can't live without seeing them.

The QuickTime VR Manager keeps track of QuickTime VR movies using an identifier called a QTVR instance (of data type QTVRInstance). Virtually all QuickTime VR Manager calls operate on QTVR instances. You can think of an instance as representing a scene -- that is, a collection of nodes, or sometimes just the current node. You get a QTVR instance by calling the QTVRGetQTVRInstance function, as shown in Listing 1. QTVRGetQTVRInstance takes a reference to a QTVR track, which you can get by calling QTVRGetQTVRTrack. (In general, you'll never do anything else to the QTVR track, so you can safely forget about it.)

Listing 1: InitApplicationWindowObject

void InitApplicationWindowObject 
                               (WindowObject theWindowObject)
{
   Track             myQTVRTrack = NULL;
   Movie             myMovie = NULL;
   MovieController   myMC = NULL;
   QTVRInstance      myInstance = NULL;
      
   if (theWindowObject == NULL)
      return;
   // make sure we can safely call the QTVR API
   if (!gQTVRMgrIsPresent)
      return;
   // find the QTVR track
   myMovie = (**theWindowObject).fMovie;
   myMC = (**theWindowObject).fController;
   myQTVRTrack = QTVRGetQTVRTrack(myMovie, 1);
   // get a QTVR instance and remember it
   QTVRGetQTVRInstance(&myInstance, myQTVRTrack, myMC);
   (**theWindowObject).fInstance = myInstance;
   // do any QTVR window configuration
   if (myInstance != NULL) {
      // set units to radians
      QTVRSetAngularUnits(myInstance, kQTVRRadians);
   }
}

The InitApplicationWindowObject function defined in Listing 1 takes as a parameter a window object, which is a handle to a data structure associated with each window containing a QuickTime (or QuickTime VR) movie:

typedef struct {
   WindowRef          fWindow;       // the window
   OSType             fObjectType;   // app-specific window tag
   Movie              fMovie;        // the main movie (QT or QTVR)
   MovieController    fController;   // the movie controller
   FSSpec             fFileFSSpec;
   short              fFileResID;
   short              fFileRefNum;
   QTVRInstance       fInstance;     // the QTVRInstance
   Handle             fAppData;      // a handle to app-specific data
} WindowObjectRecord, *WindowObjectPtr, **WindowObject;

The fields of this structure contain, among other things, references to the movie and movie controller, and the QTVR instance. The field fAppData is a handle to any other data that might have to be associated with the window. For the time being, we'll ignore that field.

Notice that Listing 1 calls the QTVRSetAngularUnits function. The QuickTime VR Manager can work with either degrees or radians when specifying angular measurements. The default angular unit type is degrees, but you can change the current unit type by calling QTVRSetAngularUnits. Internally, the QuickTime VR Manager always uses radians, and in some situations gives you measurements in radians no matter what the current angular unit. In general, therefore, we find it easier to work in radians most of the time, so we reset the angular unit type to radians. (Your preference may vary!) Of course, we can define some macros to convert degrees to radians and vice versa, should the need arise:

#define kVRPi                ((float)3.1415926535898)
#define kVR2Pi               ((float)(2.0 * 3.1415926535898))
#define QTVRUtils_DegreesToRadians(x)      \
                             ((float)((x) * kVRPi / 180.0))
#define QTVRUtils_RadiansToDegrees(x)      \
                             ((float)((x) * 180.0 / kVRPi))

Where Am I?

Finally, we're ready to use the QuickTime VR Manager to do some real work. The most basic way to use the API is to control the view parameters of a node -- the pan, tilt, and field of view angles. Listing 2 defines a function that gradually increments the pan angle through 360 degrees. With panoramas, this has the effect of making the user seem to spin a full circle (as if the user is spinning on a rotating stool). With objects, this has the effect of making the object spin around a full circle (as if the object is spinning on a lazy Susan).

Listing 2: SpinAroundOnce

void SpinAroundOnce (QTVRInstance theInstance)
{
   float      myOrigPanAngle, myCurrPanAngle;
   
   myOrigPanAngle = QTVRGetPanAngle(theInstance);
   for (myCurrPanAngle = myOrigPanAngle; 
       myCurrPanAngle <= myOrigPanAngle + kVR2Pi; 
       myCurrPanAngle += QTVRUtils_DegreesToRadians(10.0)) {
     QTVRSetPanAngle(theInstance, myCurrPanAngle);
     QTVRUpdate(theInstance, kQTVRCurrentMode);
   }
}

The idea here is simple: get the starting pan angle (by calling QTVRGetPanAngle) and then repeatedly increment the pan angle by a certain amount (here, 10 degrees) until a full circle has been traversed. Note that we must call the QTVRUpdate function after we set a new pan angle to make sure the updated view is displayed on the screen.

Overlaying Images on a Panorama

Suppose you wanted to display a logo or other graphical element in the corner of a QuickTime VR panoramic movie (in the way that some TV channels often do to discourage pirating). Because the overlaid logo isn't affected by the view settings, we can just draw it into the panorama's prescreen buffer. We must keep track of the picture to be drawn, so we'll create an application data record and store a handle to that record in the fAppData field of the window object record. For present purposes, our data record can look like this:

typedef struct {
   PicHandle   fPicture;       // the picture to display
   Boolean   fDisplayPicture;  // is the picture to be displayed?
} ApplicationDataRecord, *ApplicationDataPtr, 
   **ApplicationDataHdl;

We'll get the overlay picture from a PICT resource, in a window data initialization routine (shown in Listing 3) called by the application's InitApplicationWindowObject function.

Listing 3: VRLogo_InitWindowData

ApplicationDataHdl VRLogo_InitWindowData 
                           (WindowObject theWindowObject)
{
   ApplicationDataHdl    myAppData;
   
   myAppData = (ApplicationDataHdl)
                           NewHandleClear(sizeof(myAppData));
   if (myAppData != NULL) {
      // get the picture to overlay
      (**myAppData).fPicture = GetPicture(kPictureResID);
      
      // set initial display state
      (**myAppData).fDisplayPicture = true;
   }
   return(myAppData);
}

Also, in the application's InitApplicationWindowObject function, we must install our prescreen buffer imaging completion procedure, which is called by the QuickTime VR Manager each time the prescreen buffer is about to be copied to the screen. We can install our procedure like this:

if (QTVRGetNodeType(myInstance, kQTVRCurrentNode) 
      == kQTVRPanoramaType) 
{
   ImagingCompleteUPP   myImagingProc;

   myImagingProc = 
      NewImagingCompleteProc(VRLogo_PrescreenRoutine);
   QTVRSetPrescreenImagingCompleteProc(myInstance, 
                myImagingProc, (SInt32)theWindowObject, 0);
}

The third parameter to QTVRSetPrescreenImagingCompleteProc is an application-specific reference constant value; here we pass the window object reference, so the prescreen buffer can access the data associated with the window.

Our prescreen buffer imaging completion procedure is called after QuickTime VR has finished drawing into the prescreen buffer. When it's called, the current graphics port is set to the prescreen buffer. All we need to do is draw the picture at the appropriate spot, as shown in Listing 4.

Listing 4: VRLogo_PrescreenRoutine

pascal OSErr VRLogo_PrescreenRoutine 
  (QTVRInstance theInstance, WindowObject theWindowObject)
{
#pragma unused(theInstance)

  ApplicationDataHdl      myAppData;
  Rect                    myMovieRect;
  Rect                    myPictRect;
   
  // get the application-specific data associated with the window
  myAppData = (ApplicationDataHdl)
                GetAppDataFromWindowObject(theWindowObject);
  if (myAppData == NULL)
     return(paramErr);

  // if there is no picture to display or displaying is toggled 
  // off, just return
  if ((**myAppData).fPicture == NULL)
     return(noErr);
            
  if (!(**myAppData).fDisplayPicture)
     return(noErr);
     
  // get the current size of the movie
  GetMovieBox((**theWindowObject).fMovie, &myMovieRect);
  
  // set the size and position of the overlay rectangle
  SetRect(&myPictRect, 0, 0, 32, 32);
  OffsetRect(&myPictRect, 
             myMovieRect.right - (myPictRect.right + 5), 
             myMovieRect.bottom - (myPictRect.bottom + 5));

  // draw the picture
  DrawPicture((**myAppData).fPicture, &myPictRect);
  
  return(noErr);
}

There's nothing very complicated in this prescreen buffer imaging completion procedure. Essentially, it just figures out where in the buffer to draw the picture and then draws it.

Note that the current release of the QuickTime VR Manager maintains prescreen buffers only for panoramic nodes. It's possible, however, with a little effort, to create your own prescreen buffer for object nodes and then perform the same kind of overlays that are possible with panoramic nodes.

Integrating with Other Media

Much of the real power provided by the QuickTime VR Manager derives from the ease with which it allows you to integrate VR movies with other media, such as video, sound, and 3D objects. In a future article, we'll show how to embed QuickDraw 3D objects in a panorama. In the meantime, we'll give you a good taste of what's possible by showing how to integrate QuickTime VR and SoundSprocket, the part of Apple Game Sprockets that supports localized sounds (that is, sounds emanating from a specific location in a panorama). We'll suppose that you're already familiar with SoundSprocket, but the ideas are so simple that you can probably follow along even if you aren't.

SoundSprocket provides a virtual audio environment consisting of a single listener and one or more sound sources. The listener and the sound sources all have independent positions in 3D space. In addition, the listener and sound sources all have independent orientations in 3D space. The basic idea behind our sample application is that the listener is situated at the nodal point of the panorama (the point around which the panorama turns) and is looking at the center of the movie window. As the user interactively changes the pan and tilt angles of the panorama, the fixed locations of the sound sources change relative to the listener. (SoundSprocket doesn't actually require that the sound sources have fixed locations, but we'll keep the locations of our sources fixed, for simplicity.)

The data we need to maintain for each VR movie has this structure:

typedef struct {
   SSpListenerReference    fListener;
   SSpSourceReference      fSources      [kMaxNumSourcesPerNode];
   SndChannelPtr           fChannels   [kMaxNumSourcesPerNode];
   SndListHandle           fResources   [kMaxNumSourcesPerNode];
   float                   fPrevPanAngle;
   float                   fPrevTiltAngle;
} ApplicationDataRecord,   *ApplicationDataPtr, 
   **ApplicationDataHdl;

We're keeping track of the listener and the sound sources used by SoundSprocket, as well as a sound channel and a sound resource for each sound in the panorama. Finally, we're keeping track of the previous pan and tilt angles, which we'll compare with the current pan and tilt angles to determine whether we need to update the listener's orientation.

We'll skip over the details of setting up the virtual audio environment. What's of interest here is the way in which we translate changes in the VR movie's pan and tilt angles into changes in the listener's orientation. We do this using a prescreen buffer imaging completion procedure, not because we want to actually draw anything into the prescreen buffer, but simply because we want to be called whenever the pan or tilt angles of a panorama have changed and (therefore) a new view is about to be displayed. Our prescreen routine is shown in Listing 5.

Listing 5: VR3DSound_PrescreenRoutine

pascal OSErr VR3DSound_PrescreenRoutine 
   (QTVRInstance theInstance, WindowObject theWindowObject)
{
   float                 myPan;
   float                 myTilt;
   TQ3Vector3D         myOrientation;
   ApplicationDataHdl    myAppData;

   myAppData = (ApplicationDataHdl)
                GetAppDataFromWindowObject(theWindowObject);
   if (myAppData == NULL)
      return(paramErr);
      
   // get the current pan and tilt angles (in radians)
   myPan = QTVRGetPanAngle(theInstance);
   myTilt = QTVRGetTiltAngle(theInstance);
   
   // determine whether the pan or tilt angle has changed
   if ((myPan == (**myAppData).fPrevPanAngle) && 
         (myTilt == (**myAppData).fPrevTiltAngle))
      return(noErr);

   (**myAppData).fPrevPanAngle = myPan;
   (**myAppData).fPrevTiltAngle = myTilt;
      
   // figure out the new orientation of the listener
   myOrientation.x = -sin(myPan) * cos(myTilt);
   myOrientation.y = sin(myTilt);
   myOrientation.z = -cos(myPan) * cos(myTilt);
      
   // set the new orientation of the listener
   SSpListener_SetOrientation
               ((**myAppData).fListener, &myOrientation);
   
   // update the virtual audio environment
   VR3DSound_Update3DSoundEnv(theWindowObject);
   
   return(noErr);
}

Once again, there isn't anything very complicated here. This prescreen routine gets the current pan and tilt angles and then (using a little elementary trigonometry) converts those angles into a point in three-dimensional space. For simplicity, we've assumed that the sound sources are all located one unit away from the listener, but it would be quite trivial to remove that restriction. Then the prescreen routine sets the new orientation for the listener and updates the virtual audio environment.

The source code for the VR3DSound sample application contains (in addition to all the necessary SoundSprocket processing) code for opening sound resources and stopping a node's sounds when the user moves to a new node. That code illustrates how to use node-entering and node-leaving procedures.

Intercepting QuickTime VR Manager Functions

Suppose you want to play a sound every time the user clicks (that is, triggers) a hot spot. The easiest way to do this is to install an intercept procedure that is called each time a hot spot is triggered. The intercept procedure simply plays the sound and then returns, whereupon QuickTime VR processes the hot spot click as usual. Listing 6 shows a simple hot spot triggering intercept procedure.

Listing 6: VRSample_InterceptRoutine

pascal void VRSample_InterceptRoutine (
                      QTVRInstance theInstance, 
                      QTVRInterceptPtr theMsg, 
                      WindowObject theWindowObject, 
                      Boolean *cancel)
{
#pragma unused(theInstance, theWindowObject)

   Boolean          myCancelInterceptedProc = false;
   switch (theMsg->selector) {
      case kQTVRTriggerHotSpotSelector:
         MyPlaySound();
         break;
         
      default:
         break;
   }
   
   *cancel = myCancelInterceptedProc;
}

An intercept routine is executed whenever the intercepted routine is called, either programmatically or by a user action. (We'll show you shortly how to specify which routine or routines you want to intercept.) On entry, the QuickTime VR Manager provides three pieces of information: the relevant QTVR instance, a pointer to an intercept record, and an application-defined reference constant, which we use here to pass in the window object. The intercept record (pointed to by the theMsg parameter) has this structure:

typedef struct QTVRInterceptRecord {
   SInt32      reserved1;
   SInt32      selector;
   SInt32      reserved2;
   SInt32      reserved3;
   SInt32      paramCount;
   void        *parameter[6];
} QTVRInterceptRecord, *QTVRInterceptPtr;

For present purposes, we need inspect only the selector field, which contains a constant that indicates which intercepted routine is being called. As you can see in Listing 6, we simply look for any calls to QTVRTriggerHotSpot and call the application-defined function MyPlaySound when we get one.

You can install an intercept procedure by calling the QTVRInstallInterceptProc function, as shown in Listing 7.

Listing 7: VRSample_InstallInterceptRoutine

void VRSample_InstallInterceptRoutine (
                               QTVRInstance theInstance, 
                               WindowObject theWindowObject)
{
   QTVRInterceptUPP   myInterceptProc;
   
   myInterceptProc = 
               NewQTVRInterceptProc(VRSample_InterceptRoutine);
   
   QTVRInstallInterceptProc(theInstance, 
                             kQTVRTriggerHotSpotSelector, 
                             myInterceptProc, 
                             (SInt32)theWindowObject, 0);
}

Virtually Finished

It seems like we've barely scratched the surface of the QuickTime VR Manager, but even so we've illustrated some very powerful capabilities for managing QuickTime VR movies programmatically. We've shown how to perform basic positioning of the viewer, how to alter the displayed image by drawing into a panorama's prescreen buffer, how to link the orientation of a listener in SoundSprocket's virtual audio environment to the QuickTime VR view angles, and how to intercept some QuickTime VR Manager functions. Not bad for just over a hundred lines of code!

We mentioned at the outset that the QuickTime VR Manager allows you integrate QuickTime movies and QuickDraw 3D objects with QuickTime VR panoramas and objects. Now that you've seen how to use the VR API, and particularly how to support SoundSprocket, you can probably figure out how to do at least the QuickDraw 3D integration yourself. If not, don't despair. Just have some patience until we show you - in our next article - how to play movies and render 3D objects in a QuickTime VR panorama.

Bibliography and References

Apple Computer, Inc. Virtual Reality Programming With QuickTime VR 2.0 (1997). Cupertino, CA.

Hoddie, Peter. "Somewhere in QuickTime: Basic Movie Playback Support". develop, The Apple Technical Journal, issue 18 (June 1994), pp. 22-25. Apple Computer's Developer Press.


Tim Monroe, monroe@apple.com, is a software engineer on the QuickTime VR team, responsible for developing sample code for the new QuickTime VR C language API. In his previous life at Apple, he worked on the Inside Macintosh team.

Bryce Wolfson, bwolfson@apple.com, is also a software engineer with Apple's QuickTime VR team. He's responsible for parts of the QTVR runtime's architecture, human interface, and application interaction, and has been known to occasionally write bits of over-commented sample code.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Whitethorn Games combines two completely...
If you have ever gone fishing then you know that it is a lesson in patience, sitting around waiting for a bite that may never come. Well, that's because you have been doing it wrong, since as Whitehorn Games now demonstrates in new release Skate... | Read more »
Call of Duty Warzone is a Waiting Simula...
It's always fun when a splashy multiplayer game comes to mobile because they are few and far between, so I was excited to see the notification about Call of Duty: Warzone Mobile (finally) launching last week and wanted to try it out. As someone who... | Read more »
Albion Online introduces some massive ne...
Sandbox Interactive has announced an upcoming update to its flagship MMORPG Albion Online, containing massive updates to its existing guild Vs guild systems. Someone clearly rewatched the Helms Deep battle in Lord of the Rings and spent the next... | Read more »
Chucklefish announces launch date of the...
Chucklefish, the indie London-based team we probably all know from developing Terraria or their stint publishing Stardew Valley, has revealed the mobile release date for roguelike deck-builder Wildfrost. Developed by Gaziter and Deadpan Games, the... | Read more »
Netmarble opens pre-registration for act...
It has been close to three years since Netmarble announced they would be adapting the smash series Solo Leveling into a video game, and at last, they have announced the opening of pre-orders for Solo Leveling: Arise. [Read more] | Read more »
PUBG Mobile celebrates sixth anniversary...
For the past six years, PUBG Mobile has been one of the most popular shooters you can play in the palm of your hand, and Krafton is celebrating this milestone and many years of ups by teaming up with hit music man JVKE to create a special song for... | Read more »
ASTRA: Knights of Veda refuse to pump th...
In perhaps the most recent example of being incredibly eager, ASTRA: Knights of Veda has dropped its second collaboration with South Korean boyband Seventeen, named so as it consists of exactly thirteen members and a video collaboration with Lee... | Read more »
Collect all your cats and caterpillars a...
If you are growing tired of trying to build a town with your phone by using it as a tiny, ineffectual shover then fear no longer, as Independent Arts Software has announced the upcoming release of Construction Simulator 4, from the critically... | Read more »
Backbone complete its lineup of 2nd Gene...
With all the ports of big AAA games that have been coming to mobile, it is becoming more convenient than ever to own a good controller, and to help with this Backbone has announced the completion of their 2nd generation product lineup with their... | Read more »
Zenless Zone Zero opens entries for its...
miHoYo, aka HoYoverse, has become such a big name in mobile gaming that it's hard to believe that arguably their flagship title, Genshin Impact, is only three and a half years old. Now, they continue the road to the next title in their world, with... | Read more »

Price Scanner via MacPrices.net

B&H has Apple’s 13-inch M2 MacBook Airs o...
B&H Photo has 13″ MacBook Airs with M2 CPUs and 256GB of storage in stock and on sale for up to $150 off Apple’s new MSRP, starting at only $849. Free 1-2 day delivery is available to most US... Read more
M2 Mac minis on sale for $100-$200 off MSRP,...
B&H Photo has Apple’s M2-powered Mac minis back in stock and on sale today for $100-$200 off MSRP. Free 1-2 day shipping is available for most US addresses: – Mac mini M2/256GB SSD: $499, save $... Read more
Mac Studios with M2 Max and M2 Ultra CPUs on...
B&H Photo has standard-configuration Mac Studios with Apple’s M2 Max & Ultra CPUs in stock today and on Easter sale for $200 off MSRP. Their prices are the lowest available for these models... Read more
Deal Alert! B&H Photo has Apple’s 14-inch...
B&H Photo has new Gray and Black 14″ M3, M3 Pro, and M3 Max MacBook Pros on sale for $200-$300 off MSRP, starting at only $1399. B&H offers free 1-2 day delivery to most US addresses: – 14″ 8... Read more
Department Of Justice Sets Sights On Apple In...
NEWS – The ball has finally dropped on the big Apple. The ball (metaphorically speaking) — an antitrust lawsuit filed in the U.S. on March 21 by the Department of Justice (DOJ) — came down following... Read more
New 13-inch M3 MacBook Air on sale for $999,...
Amazon has Apple’s new 13″ M3 MacBook Air on sale for $100 off MSRP for the first time, now just $999 shipped. Shipping is free: – 13″ MacBook Air (8GB RAM/256GB SSD/Space Gray): $999 $100 off MSRP... Read more
Amazon has Apple’s 9th-generation WiFi iPads...
Amazon has Apple’s 9th generation 10.2″ WiFi iPads on sale for $80-$100 off MSRP, starting only $249. Their prices are the lowest available for new iPads anywhere: – 10″ 64GB WiFi iPad (Space Gray or... Read more
Discounted 14-inch M3 MacBook Pros with 16GB...
Apple retailer Expercom has 14″ MacBook Pros with M3 CPUs and 16GB of standard memory discounted by up to $120 off Apple’s MSRP: – 14″ M3 MacBook Pro (16GB RAM/256GB SSD): $1691.06 $108 off MSRP – 14... Read more
Clearance 15-inch M2 MacBook Airs on sale for...
B&H Photo has Apple’s 15″ MacBook Airs with M2 CPUs (8GB RAM/256GB SSD) in stock today and on clearance sale for $999 in all four colors. Free 1-2 delivery is available to most US addresses.... Read more
Clearance 13-inch M1 MacBook Airs drop to onl...
B&H has Apple’s base 13″ M1 MacBook Air (Space Gray, Silver, & Gold) in stock and on clearance sale today for $300 off MSRP, only $699. Free 1-2 day shipping is available to most addresses in... Read more

Jobs Board

Medical Assistant - Surgical Oncology- *Apple...
Medical Assistant - Surgical Oncology- Apple Hill Location: WellSpan Medical Group, York, PA Schedule: Full Time Sign-On Bonus Eligible Remote/Hybrid Regular Apply Read more
Omnichannel Associate - *Apple* Blossom Mal...
Omnichannel Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Cashier - *Apple* Blossom Mall - JCPenney (...
Cashier - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Blossom Mall Read more
Operations Associate - *Apple* Blossom Mall...
Operations Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Business Analyst | *Apple* Pay - Banco Popu...
Business Analyst | Apple PayApply now " Apply now + Apply Now + Start applying with LinkedIn Start + Please wait Date:Mar 19, 2024 Location: San Juan-Cupey, PR Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.