TweetFollow Us on Twitter

Using File Manager From MP Tasks

Volume Number: 14 (1998)
Issue Number: 8
Column Tag: Toolbox Techniques

Using the File Manager from MP Tasks

by by Matthew Xavier Mora
Edited by Peter N Lewis

How to get data in and out of your MP Task

One of the most common complaints I received while supporting MP Library in Developer Technical Support was that you could not call the toolbox from an MP Task. Multiple preemptive tasks are not much use if you cannot get data into and out of them efficiently. This article shows one way to get data into and out of an MP Task using the file manager, however the techniques used here can be modified for other I/O operations (like audio, video or networking). "But, I thought you couldn't call the file manager from MP Tasks?" Well, you thought wrong. :-) Read on...

Background

In the early version of the MP library there was no easy way to call the toolbox because the MP Library was designed to be compatible with Copland's kernel tasking model. Since the Mac OS toolbox wasn't going to be available from Copland's kernel tasks, the same was done for the Mac OS version of the MP Library. After the Copland project was canceled it was decided to publish a few previously undocumented routines that let you work with the Mac OS toolbox from a task. One of the routines published is MPRPC. MPRPC is a remote procedure mechanism that lets you specify a routine to execute at a time when it is safe to make toolbox calls. It does this by suspending the task and then executing the supplied routine during SystemTask() time. The task is suspended until MPYield is called or until any toolbox routine calls SystemTask(). MPRPC is used internally in the MP Library to implement calls such as MPAllocate and MPAllocateSys (which is why these are blocking calls).

The code in this article is based on the MP File Library that I wrote before the MPRPC call was published. The MP File Library used MPQueues to communicate with the main task and have it execute toolbox commands.

Review

Let's review some of the MP programming guidelines and how adding blocking calls can change some of these guidelines.

  1. Your tasks should do a considerable amount of work. If not, the benefits of using MP will be lost in the overhead of the scheduler and task switching. Adding blocking calls to your tasks adds additional overhead. The main benefit here is that by being able to call the toolbox from an MP Task your task can run autonomously from the main application thread. This results in a better user interface response from the application since the application can off load a time consuming task and call the main event loop more often giving the blocking calls more time to execute the toolbox calls.
  2. You should allocate no more than (MPProcessors() - 1) number of tasks. While it is important to keep the number of tasks low so that task switching does not impact performance, adding blocking calls to a task will also hurt performance if nothing calls MPYield(). "Wait, I thought MP Tasks were preemptive?" Yes they are but if the task is blocked waiting on a resource, the resource can't be released until the main thread calls WaitNextEvent() or another task calls MPYield(). That being the case, if you use MPRPC calls it is a good idea to bend the n-1 rule and create an extra task that can help unblock any waiting tasks.
  3. You should use MPQueues or MPSemaphores when communicating with MP Tasks. This does not change if you are using MPRPC so you should heed this warning.

Get On With It

OK, so how do I call the File Manager? For this simple example I will implement five MP calls that duplicate FSpOpenDF, FSClose, FSRead, FSWrite and SetFPos. Those are all the calls we need for a simple demo. We'll start with a FSRead type call.

First lets define a structure to pass to the MPRPC callback routines that will hold the values that we need to handle all the File Manager calls.

typedef struct FSParamRec{
   short        refNum;        // file ref num
   long         count;         // for read
   Ptr          buffPtr;       // for read
   FSSpecPtr    spec;          // for open
   short        permission;    // for open
   short        posMode;       // for setfpos
   long         posOff;        // for setfpos   
   OSErr        result;        // error result
} FSParamRec,*FSParamRecPtr;

Now lets implement the callback routine that gets called at main application time. This routine will be executing at SystemTask time which means you can call any toolbox routine except for any routines that might call SystemTask() again.

static void * FSReadCallBack( void * parameter)
{
   FSParamRecPtr fsprp = (FSParamRecPtr)parameter;
   
   if (fsprp != nil) {
      fsprp->result = FSRead(fsprp->refNum,
                   &fsprp->count,
                   fsprp->buffPtr);
   }
   
   return fsprp;
}

First we check to make sure the parameter that was passed in is not nil then we simply call the File manager's FSRead call. When FSRead returns, we put the result into the result field and then return the pointer to the struct that was passed in.

All that is left is to do is to implement the new MyMPFSRead call.

pascal OSErr MyMPFSRead(short refNum,
                         long * count,
                         void * buffPtr)
{
   FSParamRec fsrr;   // make the record on the stack 
                      // no worries since it is a blocking call

   fsrr.refNum   = refNum;
   fsrr.count    = *count;
   fsrr.buffPtr  = buffPtr;
   fsrr.result   = paramErr;   //preset in case 
                               //anything goes wrong
   
   (void) _MPRPC(FSReadCallBack,&fsrr); 
   //ignore what is returned


   *count = fsrr.count;   //return the new count

   return fsrr.result;    //return the result
}

First we allocate a FSReadRec on the stack that gets passed to MPRPC. We fill out the fields in the struct with what was passed into us, call MPRPC and wait for the result. Then return the result to the caller.

That's it. You can now call FSRead from an MP Task. Using the same basic techniques you can implement all the file manager calls you need to get data in and out of your tasks. Now lets see how the task calls the new routines.

The MP Task itself is pretty straight forward as a result of the blocking I/O calls since there are no flags or spin loops to worry about.

static long MyMPTask(void * param)
{
   FSSpecPtr   fsp;
   Boolean     done = false;
   OSStatus    status;
   OSErr       err;
   MPQueueID   mpq = (MPQueueID) param;

   
   // don't start until we get the message
   status = MPWaitOnQueue(mpq,&fsp,nil,nil, kDurationForever);
   // the message is the file spec
   if (fsp) {   
      short          refNum;
      long          count = 1024; //read 1k of data
      
      err = MyMPFSpOpenDF(fsp,fsRdPerm,&refNum);
      if (!err) {
         err = MyMPSetFPos(refNum,fsFromStart,0);
         if (!err) {
   
#if qUseAsyncRead         
                err = MyMPFSReadAsync(refNum,&count,gBuffer);
#else
                err = MyMPFSRead(refNum,&count,gBuffer);
#endif            
            // we got some data. you could compress it
            // do FFT's on it or whatever.
            // In our case we just set the flag that we got
            // the data and tell the processors to sync up
            
            if (count > 0) {   
               gCount = count; // signal that we got some text   
               __eieio();      // sync processors
            } else {
               gCount = -1;    // signal that we got an error   
               __eieio();      // sync processors
            }
         } 
         err = MyMPFSClose(refNum);
      }
   }
}

In our task we immediately block (as every task should) on MPWaitOnQueue waiting for the FSSpecPtr from the application. When MPWaitOnQueue returns, we check the file spec pointer to make sure it is not nil and precedes to open the file. We set the file position to the beginning of the file and start the read operation. Notice that for either the async or non async case the code is still the same. The only difference is to the application since the task is blocked until the read completes. After the read completes, this is where you would do some serious processing on the data. It is very important that you do a lot of processing to minimize the overhead of the blocking I/O calls. The demo doesn't do any processing so the next thing to do is to set the gCount variable indicating we got the data making sure the write get synchronized with the other processors. We close the file and return. Returning from the task kills the task. You might want the task to hang around and be ready to process another file. In that case set up a while loop on MPWaitOnQueue. You can set the exit termination condition to be a nil FSSpecPtr.

Adding More Features

The FSRead technique is good at getting data in and out of your task but you basically block the entire application while it waits for the FSRead to complete. We can improve this by using asynchronous file manager calls to keep from blocking the main application task while executing a Read call.

We need a different structure to do an async read. I wrap the new struct around a ParamBlockRec to contain the flag needed to signal the completion of the read call.

typedef struct FSReadAsyncRec { ParamBlockRec pb; // standard paramblock Boolean callPending; // our pending flag } FSReadAsyncRec, *FSReadAsyncRecPtr;

The MyMPFSReadAsync code is a little more complicated but it saves having to have another task running just to call MPYield() since this routine spins on MPYield waiting for the PBRead to complete.

static pascal OSErr MyMPFSReadAsync(short refNum,
                               long * count,
                               void * buffPtr)
{
   FSReadAsyncRec       fsrar;      // make the rec on the stack 
   // Build a rountine descriptor by hand since we can't call
   // NewIOCompletionProc(userRoutine)
   RoutineDescriptor    ioCompProc = 
                  BUILD_ROUTINE_DESCRIPTOR(uppIOCompletionProcInfo,
                                          MyReadCompletion);

   ClearBlock(&fsrar,sizeof(fsrar)); 
   
   fsrar.pb.ioParam.ioRefNum     =   refNum;
   fsrar.pb.ioParam.ioReqCount   =   *count;
   fsrar.pb.ioParam.ioBuffer     =   buffPtr;
   fsrar.pb.ioParam.ioCompletion =   &ioCompProc;
   fsrar.callPending             = true;
   __eieio();                  //ensure that callPending gets set
                               //before we call MPRPC
   
   (void) _MPRPC(FSReadAsyncCallBack,&fsrar); //ignore what is 

   // spin waiting for flag to be set in completionRoutine

   while ( fsrar.callPending ) { //Spin waiting for completion
      MPYield();
   }

   *count = fsrar.pb.ioParam.ioActCount;   
                  //return the new count

   return fsrar.pb.ioParam.ioResult;   //return the result
}

MyMPFSReadAsync sets up the parameter block, builds a completion routine descriptor on the fly, calls MPRPC and then spins in a tight loop calling MPYield until the callPending flag is cleared.

The FSReadAsyncCallBack routine is very simple.

static void * FSReadAsyncCallBack( void * parameter)
{
   FSReadAsyncRecPtr fsr = (FSReadAsyncRecPtr) parameter;
   OSErr err;
   
   if (fsr != nil) {   
      err = PBReadAsync((ParmBlkPtr)fsr);   
                           //just call PBRead and return
   }                      // completion routine sets the flag   
   return fsr;
}

FSReadAsyncCallBack just calls PBReadAsync and returns. Below is the completion routine that tells the task the read has completed.

static void MyReadCompletion(ParmBlkPtr pb)
{
   FSReadAsyncRecPtr fs = (FSReadAsyncRecPtr)pb; 
   
   fs->callPending = false;  // set flag
   __eieio();                   // make sure it sticks
}

It just sets the callPending flag, signals the processors to sync up and returns. We can't set a MPQueue or a MPSemapore in here (which would be the better way to do it) because MP Library calls can't be called at interrupt time.

Handling asynchronous routines gets a little more complicated but it saves having to make sure other tasks are running just to call MPYield(). Now you might be thinking why are we using a flag when you could just spin on ioResult? Read on to see why this is not good idea...

Gotchas

When working with multiple processors some conventional Mac programming wisdom goes out the window. A good case in point is when ioResult is set. Normally ioResult is set to 1 to indicate a call is pending. The last thing the file manager does before calling the ioCompletion routine is to set ioResult to the error result from the parameter block call. None of this really changes when multiple processors are involved but the non-main processors are not bound by the 68k enable/disable interrupt tricks. So if your MP Task spins on ioResult waiting to see when the read is complete (ioResult != 1) your task starts to execute before the file manager is done with the parameter block. After the file manager sets the ioResult field, it gets the ioCompletion routine's address from the parameter block and jumps to it.

In our case the parameter block in on the stack and when the task unblocks, the stack is released and your task crunches merrily along where a parameter block used to be (and is still in use by the file manager). The second processor could be a 200 MHz CPU and in the time the file manager has set ioResult and jumps to the completion routine, your task could be millions of instructions away using the memory where the parameter block used to be.

The same is true for many of parts of the Mac OS Toolbox. The critical region technique of disabling interrupts does not work well when multiple processors are involved. So be careful and always use MPQueues, MPSemaphores and MPCriticalRegions to coordinate your various tasks.

Another gotcha may be in your thought process. You might be thinking that it would be cool to use the same techniques mentioned in the article to make every Toolbox call available from MP tasks. While this is possible, and would make your task code a lot easier to write, it is not a good idea. The benefits of multiprocessing only come from careful algorithm design, implementation, and profiling. Guideline #1 mentioned above says that your task should do a considerable amount of work to gain any performance improvements. Having your task block, waiting on a bunch of toolbox calls is not going to improve performance. On the other hand having to load all the data you need into memory before your task can start running may not be feasible either. This is where a careful balance of having main processor moving data in and out of your task while processors n+1 crunch along can really pay off.

More MP Information

Hopefully, this article piqued your interest in Multiprocessing. If you want more information there are a number of documents and resources to help you get the most out of MP. An introduction to MP systems was printed in MacTech March '96, TechNote 1071 on Multiprocessing is on the web http://www.apple.com/developer/ and I have set up a MP mailing list where developers can ask questions on MP programming issues. The list includes folks like the senior engineer who wrote the MP Library as well as Chris Cooksey and myself. For subscription information you can go to my web site http://www.best.com/~mxmora/mxm.html. Also, don't forget Apple Developer Technical Support is there for information about MP's past, present and future.

Summary

I hope this article shows how easy it is to get data into and out of your MP tasks. Use this information wisely and you should see some real improvements in your applications performance. You can use these techniques to work with other I/O technologies like networking, graphics and sound. I have created a MP File Library that you may want to use based on some of the techniques used in this article. It uses a slightly more complicated model for better performance. You can download a copy of my MP File Library from my web site at http://www.best.com/~mxmora/software.html. Good luck, and happy multiprocessing.


Matthew Xavier Mora was the engineer responsible for answering questions on Multitasking support in Apple's Developer Technical Support. As a self proclaimed evangelist for the Multi-processing API library he was instrumental in convincing both third-party developers and Apple engineers to implement MP support in their software. If you were ever thinking about moving into the Silicon Valley, consider that this article was written while Matt was sitting all night outside a school building waiting to register his son for pre-school. When Matt is not out doing crazy things like that you can reach him at mxmora@best.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Drop to GIF 1.28 - Drag-and-drop creatio...
Drag to GIF means zero-click movie-to-GIF conversion. Select a folder to watch and every movie saved or moved into that folder will be converted to an animated GIF. Features Dragging - Drag a movie... Read more
BetterTouchTool 3.158 - Customize multi-...
BetterTouchTool adds many new, fully customizable gestures to the Magic Mouse, Multi-Touch MacBook trackpad, and Magic Trackpad. These gestures are customizable: Magic Mouse: Pinch in / out (zoom)... Read more
NeoFinder 7.4.1 - Catalog your external...
NeoFinder (formerly CDFinder) rapidly organizes your data, either on external or internal disks, or any other volumes. It catalogs and manages all your data, so you stay in control of your data... Read more
Tunnelblick 3.8.0 - GUI for OpenVPN.
Tunnelblick is a free, open source graphic user interface for OpenVPN on OS X. It provides easy control of OpenVPN client and/or server connections. It comes as a ready-to-use application with all... Read more
Tweetbot 3 for Twitter 3.3.1 - Popular T...
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
Monosnap 3.5.9 - Versatile screenshot ut...
Monosnap lets you capture screenshots, share files, and record video and .gifs. Features Capture Capture full screen, just part of the screen, or a selected window Make your crop area pixel... Read more
Monosnap 3.5.9 - Versatile screenshot ut...
Monosnap lets you capture screenshots, share files, and record video and .gifs. Features Capture Capture full screen, just part of the screen, or a selected window Make your crop area pixel... Read more
Tweetbot 3 for Twitter 3.3.1 - Popular T...
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
Tunnelblick 3.8.0 - GUI for OpenVPN.
Tunnelblick is a free, open source graphic user interface for OpenVPN on OS X. It provides easy control of OpenVPN client and/or server connections. It comes as a ready-to-use application with all... Read more
Bookends 13.2.5 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Bookends uses the cloud to sync reference libraries on all the Macs you use.... Read more

Latest Forum Discussions

See All

Void Tyrant guide - Tips and tricks for...
Void Tyrant continues to get a lot of play in these parts. Probably because the game is just so deep and varied. The next stop on our guide series for Void Tyrant is class-specific guides. First up is the Knight, as it’s the first class anyone has... | Read more »
Summon beasts and battle evil in epic re...
Imagine a tale of conlict between factions of good and evil, where rogueish heroes summon beasts to aid them in them in warfare and courageously battle dragons over fields of scorched earth and brimstone - that's exactly the essence of epic fantasy... | Read more »
Upcoming visual novel Arranged shines a...
If you’re in the market for a new type of visual novel designed to inform and make you think deeply about its subject matter, then Arranged by Kabuk Games could be exactly what you’re looking for. It’s a wholly unique take on marital traditions in... | Read more »
TEPPEN guide - The three best decks in T...
TEPPEN’s unique take on the collectible card game genre is exciting. It’s just over a week old, but that isn’t stopping lots of folks from speculating about the long-term viability of the game, as well as changes and additions that will happen over... | Read more »
Intergalactic puzzler Silly Memory serve...
Recently released matching puzzler Silly Memory is helping its fans with their intergalactic journeys this month with some very special offers on in-app purchases. In case you missed it, Silly Memory is the debut title of French based indie... | Read more »
TEPPEN guide - Tips and tricks for new p...
TEPPEN is a wild game that nobody asked for, but I’m sure glad it exists. Who would’ve thought that a CCG featuring Capcom characters could be so cool and weird? In case you’re not completely sure what TEPPEN is, make sure to check out our review... | Read more »
Dr. Mario World guide - Other games that...
We now live in a post-Dr. Mario World world, and I gotta say, things don’t feel too different. Nintendo continues to squirt out bad games on phones, causing all but the most stalwart fans of mobile games to question why they even bother... | Read more »
Strategy RPG Brown Dust introduces its b...
Epic turn-based RPG Brown Dust is set to turn 500 days old next week, and to celebrate, Neowiz has just unveiled its biggest and most exciting update yet, offering a host of new rewards, increased gacha rates, and a brand new feature that will... | Read more »
Dr. Mario World is yet another disappoin...
As soon as I booted up Dr. Mario World, I knew I wasn’t going to have fun with it. Nintendo’s record on phones thus far has been pretty spotty, with things trending downward as of late. [Read more] | Read more »
Retro Space Shooter P.3 is now available...
Shoot-em-ups tend to be a dime a dozen on the App Store, but every so often you come across one gem that aims to shake up the genre in a unique way. Developer Devjgame’s P.3 is the latest game seeking to do so this, working as a love letter to the... | Read more »

Price Scanner via MacPrices.net

13″ 1.8GHz MacBook Air on sale again for only...
Amazon has the 2017 13″ 1.8GHz/128GB MacBook Air on sale today for only $749.99 shipped. That’s $250 off Apple’s original MSRP for this model and the cheapest MacBook Air available from any Apple... Read more
Apple’s $1489 clearance price on refurbished...
Apple has Certified Refurbished 2018 13″ 2.3GHz 4-Core Touch Bar MacBook Pros available starting at $1489. Apple’s one-year warranty is included, shipping is free, and each MacBook has a new outer... Read more
New 2019 13″ 2.4GHz 4-Core MacBook Pros on sa...
Apple resellers B&H Photo and Amazon are offering the new 2019 13″ 2.4GHz 4-Core Touch Bar MacBook Pros for $150 off Apple’s MSRP. These are the same MacBook Pros sold by Apple in its retail and... Read more
B&H drops prices another $50 on clearance...
B&H Photo has dropped prices on clearance 2018 13″ MacBook Airs by a further $50 with models now available for $250 off Apple’s original MSRP. Overnight shipping, or expedited shipping, is free... Read more
Find the best sales & lowest prices on Ap...
Our Apple award-winning price trackers are the best place to look for the best sales and lowest prices on Apple gear. Scan our price trackers for the latest information on sales, bundles, and... Read more
Apple has clearance 2018 13″ MacBook Airs now...
Apple has Certified Refurbished 2018 13″ MacBook Airs available starting at only $849. Each MacBook features a new outer case, comes with a standard Apple one-year warranty, and is shipped free. The... Read more
Save $400 on the 8-Core iMac Pro today at Ama...
Amazon has the base 8-core iMac Pro on sale today for $4599 including free shipping. Their price is $400 off Apple’s MSRP, and it’s the currently lowest price available for an iMac Pro. For the... Read more
Flash sale! New 11″ 1TB WiFi iPad Pros for th...
Amazon has the 11″ 1TB WiFi iPad Pro on sale today for only $1199.99 including free shipping. Their price is $350 off Apple’s MSRP for this model, and it’s the lowest price ever for a 1TB 11″ iPad... Read more
Weekend Deal: 2018 13″ MacBook Airs starting...
B&H Photo has clearance 2018 13″ MacBook Airs available starting at only $999 with all models now available for $200 off Apple’s original MSRP. Overnight shipping, or expedited shipping, is free... Read more
Apple has clearance 10.5″ iPad Pros available...
Apple has Certified Refurbished 2017 10.5″ iPad Pros available starting at $469. An Apple one-year warranty is included with each iPad, outer shells are new, and shipping is free: – 64GB 10″ iPad Pro... Read more

Jobs Board

Best Buy *Apple* Computing Master - Best Bu...
**711495BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Store Associates **Location Number:** 000882-Waterfront-Store **Job Description:** The Read more
Best Buy *Apple* Computing Master - Best Bu...
**711597BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000873-Colma CA-Store **Job Description:** **What does a Read more
Best Buy *Apple* Computing Master - Best Bu...
**711346BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Store Associates **Location Number:** 001095-Chesterfield-Store **Job Description:** Read more
Best Buy *Apple* Computing Master - Best Bu...
**704899BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000135-Pleasant Hill-Store **Job Description:** **What does Read more
Best Buy *Apple* Computing Master - Best Bu...
**707083BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000045-Rockford-Store **Job Description:** **What does a Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.