A Quick Trip Into the Depths

ResError Considered Harmful?

by Steve Jasik, Menlo Park, CA, macnosy@netcom.com

Nosy is over 10 years old, having been introduced to the world in Nov ’84 and first shipped in Jan ’85. It has been a few years since I have written an article for MacTech. Since ’84, the Mac has become more powerful than the mainframes I used to work on, and the system has advanced in utility and complexity. Nosy hasn’t changed too much, and some pseudo alternatives to it, such as the ResEdit CODE Editor have come along, but I used it recently to get the global view of a piece of the ROM.

The particular example I’ll show here came to me as a bug in my debugger that showed up on a IIfx running System 7.5. After some investigation, it turned out that the problem was some less-than-robust code in Sound Manager 3.0. The crux of the problem was that, despite what Inside Macintosh or today’s equivalent of it says, the Resource Manager doesn’t always return a non-zero value of ResErr when it doesn’t find a requested resource. More on this later.

The original problem presented to me was that a user was trying to debug an INIT and my debugger was barfing. To find the bug, I set up my normal debugger bug-finding environment, in which I run two copies of my Debugger.

The first one debugs the second one, and if both fail, I have Macsbug running “behind” the first one. For the curious, the magic behind having one debugger debug a second copy of itself is to do a complete context switch of low memory, which includes the exception vectors (0-$FF) and the Macintosh globals ($100-$1E00) when entering The Debugger.

Using this method, I found that someone was trying to do a _CloseResFile on me. It took a few hours to trace it back into the Sound Manager.

At this point let me mention a trick I have used over the years to get a better grip on where one is in a random blowup. One of the most important pieces of information is the stack crawl. It answers the question of how and sometimes why the program is where it is. Many times the stack crawl is less than informative, and those who use Macsbug know well that the ROM part of it can be useless. Given this, how can we recover any information about how the program got to the point it is currently at? The basic answer is to try and repair the current address/bus error by fixing up a register or so (e.g. by putting $0 or $40801000 into the register which has a bus error value that the current instruction is attempting to read from or write to), advancing the Program Counter, and then carefully single stepping our way out of the current procedure and ‘up the stack’ so we can get an idea of where we came from.

In this case, this method worked for me, and I found myself in Sound Manager code, so the next problem was to get a decent disassembly of it so I could figure out why it was trying to do a CloseResFile.

After some disassembly work in Nosy, I found myself staring at the following two procedures, one which I have named open_Snd_Prefs :

open_Snd_Prefs

   B1B10: 4E56 FFB4      'NV..'           LINK    A6,#-$4C
   B1B14: 48E7 0308      'H...'           MOVEM.L D6-D7/A4,-(A7)
   B1B18: 598F           'Y.'             SUBQ.L  #4,A7
   B1B1A: 2F3C 5354 5220 '/<STR '         PUSH.L  #'STR '
   B1B20: 3F3C BF48      '?<.H'           PUSH    #$BF48
   B1B24: A9A0           '..'             _GetResource 
 ; (theType:ResType; ID:INTEGER):Handle 
   B1B26: 285F           '(_'             POP.L   A4
   B1B28: 200C           ' .'             MOVE.L  A4,D0
   B1B2A: 660A           10B1B36          BNE.S   mnn_1
   B1B2C: 558F           'U.'             SUBQ.L  #2,A7
   B1B2E: A9AF           '..'             _ResError ; :OSErr 
   B1B30: 3E1F           '>.'             POP     D7     
; ROM returns ResErr = 0 !! *****
   B1B32: 6000 0094      10B1BC8          BRA     mnn_4 
no error from RsrcMgr, so we exit with D0 = 0, but we haven't opened 
the Prefs file!!!! 
 
   B1B36: 558F           'U.'    mnn_1    SUBQ.L  #2,A7
   B1B38: 3F3C 8000      '?<..'           PUSH    #$8000
   B1B3C: 2F3C 7072 6566 '/<pref'         PUSH.L  #'pref'
   B1B42: 7001           'p.'             MOVEQ   #1,D0
   B1B44: 1F00           '..'             PUSH.B  D0
   B1B46: 486E FFFE      200FFFE          PEA     wnn_4(A6)
   B1B4A: 486E FFFA      200FFFA          PEA     wnn_3(A6)
   B1B4E: 7000           'p.'             MOVEQ   #0,D0
   B1B50: A823           '.#'             _AliasMgr 
; (D0/selector:INTEGER) 
   B1B52: 3E1F           '>.'             POP     D7
   B1B54: 6672           10B1BC8          BNE.S   mnn_4
   B1B56: 204C           ' L'             MOVEA.L A4,A0
   B1B58: A029           '.)'             _HLock  ; (A0/h:Handle) 
   B1B5A: 558F           'U.'             SUBQ.L  #2,A7
   B1B5C: 3F2E FFFE      200FFFE          PUSH    wnn_4(A6)
   B1B60: 2F2E FFFA      200FFFA          PUSH.L  wnn_3(A6)
   B1B64: 2F14           '/.'             PUSH.L  (A4)
   B1B66: 486E FFB4      200FFB4          PEA     wnn_2(A6)
   B1B6A: 303C 0001      '0<..'           MOVE    #1,D0
   B1B6E: AA52           '.R'             _HighLvlFSDisptch 
;(D0/selector:INTEGER) 
   B1B70: 3E1F           '>.'             POP     D7
   B1B72: 0C47 FFD5      '.G..'           CMPI    #-43,D7
   B1B76: 6622           10B1B9A          BNE.S   mnn_2
   B1B78: 4A2E 000B      200000B          TST.B   param2(A6)
   B1B7C: 671C           10B1B9A          BEQ.S   mnn_2
   B1B7E: 486E FFB4      200FFB4          PEA     wnn_2(A6)
   B1B82: 2F3C 7361 6420 '/<sad '         PUSH.L  #'sad '
   B1B88: 2F3C 7072 6566 '/<pref'         PUSH.L  #'pref'
   B1B8E: 70FF           'p.'             MOVEQ   #-1,D0
   B1B90: 3F00           '?.'             PUSH    D0
   B1B92: 303C 000E      '0<..'           MOVE    #14,D0
   B1B96: AA52           '.R'             _HighLvlFSDisptch
   B1B98: 4247           'BG'             CLR     D7
   B1B9A: 4A47           'JG'    mnn_2    TST     D7
   B1B9C: 662A           10B1BC8          BNE.S   mnn_4
   B1B9E: 558F           'U.'             SUBQ.L  #2,A7
   B1BA0: 486E FFB4      200FFB4          PEA     wnn_2(A6)
   B1BA4: 7003           'p.'             MOVEQ   #3,D0
   B1BA6: 1F00           '..'             PUSH.B  D0
   B1BA8: 303C 000D      '0<..'           MOVE    #13,D0
   B1BAC: AA52           '.R'             _HighLvlFSDisptch
   B1BAE: 3C1F           '<.'             POP     D6
   B1BB0: 0C46 FFFF      '.F..'           CMPI    #-1,D6
   B1BB4: 6608           10B1BBE          BNE.S   mnn_3
   B1BB6: 558F           'U.'             SUBQ.L  #2,A7
   B1BB8: A9AF           '..'             _ResError ; :OSErr 
   B1BBA: 3E1F           '>.'             POP     D7
   B1BBC: 600A           10B1BC8          BRA.S   mnn_4
   B1BBE: 206E 000C      200000C mnn_3    MOVEA.L param1(A6),A0
   B1BC2: 3086           '0.'             MOVE    D6,(A0)
   B1BC4: 7000           'p.'             MOVEQ   #0,D0
   B1BC6: 6002           10B1BCA          BRA.S   mnn_5 
   B1BC8: 3007           '0.'    mnn_4    MOVE    D7,D0
   B1BCA: 4CEE 10C0 FFA8 200FFA8 mnn_5    
MOVEM.L wnn_1(A6),D6-D7/A4
   B1BD0: 4E5E           'N^'             UNLK    A6
   B1BD2: 4E75           'Nu'             RTS       
  
 
 
   B1CA6: 4E56 FFFE      'NV..'  proc4742 LINK    A6,#-2
   B1CAA: 48E7 0738      'H..8'           
MOVEM.L D5-D7/A2-A4,-(A7)
   B1CAE: 246E 000C      200000C     MOVEA.L param2(A6),A2
   B1CB2: 266E 0008      2000008     MOVEA.L param3(A6),A3
   B1CB6: 7000           'p.'             MOVEQ   #0,D0
   B1CB8: 1012           '..'             MOVE.B  (A2),D0
   B1CBA: 4A80           'J.'             TST.L   D0
   B1CBC: 6604           10B1CC2          BNE.S   mnq_1
   B1CBE: 7ECE           '~.'             MOVEQ   #-50,D7
   B1CC0: 6074           10B1D36          BRA.S   mnq_6
   B1CC2: 200B           ' .'    mnq_1    MOVE.L  A3,D0
   B1CC4: 6604           10B1CCA          BNE.S   mnq_2
   B1CC6: 7ECE           '~.'             MOVEQ   #-50,D7
   B1CC8: 606C           10B1D36          BRA.S   mnq_6
         
   B1CCA: 558F           'U.'    mnq_2    SUBQ.L  #2,A7
   B1CCC: A994           '..'             _CurResFile ; :RefNum 
   B1CCE: 3C1F           '<.'             POP     D6
   B1CD0: 486E FFFE      200FFFE          PEA     wnq_2(A6)
   B1CD4: 7000           'p.'             MOVEQ   #0,D0
   B1CD6: 2F00           '/.'             PUSH.L  D0
   B1CD8: 4EBA FE36      10B1B10          JSR  open_Snd_Prefs
   B1CDC: 3E00           '>.'             MOVE    D0,D7
   B1CDE: 504F           'PO'             ADDQ    #8,A7
   B1CE0: 670A           10B1CEC          BEQ.S   mnq_3
; if no error then get 'sysb' resource 
   B1CE2: 3F06           '?.'             PUSH    D6
   B1CE4: A998           '..'             _UseResFile 
; (frefNum:RefNum) 
   B1CE6: 3D47 0014      2000014          MOVE D7,funRslt(A6)
   B1CEA: 604E           10B1D3A          BRA.S   mnq_7 
 
   B1CEC: 598F           'Y.'    mnq_3    SUBQ.L  #4,A7
   B1CEE: 2F2E 0010      2000010          PUSH.L  param1(A6)
   B1CF2: 2F0A           '/.'             PUSH.L  A2
   B1CF4: A820           '. '             _Get1NamedResource
 ; (theType:ResType; name:Str255):Handle 
   B1CF6: 285F           '(_'             POP.L   A4
   B1CF8: 558F           'U.'             SUBQ.L  #2,A7
   B1CFA: A9AF           '..'             _ResError ; :OSErr 
   B1CFC: 3E1F           '>.'             POP     D7
   B1CFE: 200C           ' .'             MOVE.L  A4,D0
   B1D00: 660A           10B1D0C          BNE.S   mnq_4    
; oops, didn't get it, close the res file
   B1D02: 4A47           'JG'             TST     D7
   B1D04: 6626           10B1D2C          BNE.S   mnq_5
   B1D06: 3E3C FF40      '><.@'           MOVE    #$FF40,D7
   B1D0A: 6020           10B1D2C          BRA.S   mnq_5
 
   B1D0C: 204C           ' L'    mnq_4    MOVEA.L A4,A0
   B1D0E: A025           '.%'             _GetHandleSize 
;(A0/h:Handle):D0\Size 
   B1D10: 2A00           '*.'             MOVE.L  D0,D5
   B1D12: 3E38 0220         $220          MOVE    MemErr,D7
   B1D16: 6614           10B1D2C          BNE.S   mnq_5
   B1D18: 204B           ' K'             MOVEA.L A3,A0
   B1D1A: 2005           ' .'             MOVE.L  D5,D0
   B1D1C: A024           '.$'             _SetHandleSize 
;(A0/h:Handle; D0/newSize:Size) 
   B1D1E: 3E38 0220         $220          MOVE    MemErr,D7
   B1D22: 6608           10B1D2C          BNE.S   mnq_5
   B1D24: 2054           ' T'             MOVEA.L (A4),A0
   B1D26: 2253           '"S'             MOVEA.L (A3),A1
   B1D28: 2005           ' .'             MOVE.L  D5,D0
   B1D2A: A22E           '..'             _BlockMove   
;(A0/srcPtr,A1/destPtr:Ptr;
   B1D2C: 3F2E FFFE      200FFFE mnq_5    PUSH    wnq_2(A6)
   B1D30: A99A           '..'             _CloseResFile 
; (refNum:RefNum) 
   B1D32: 3F06           '?.'             PUSH    D6
   B1D34: A998           '..'             _UseResFile 
; (frefNum:RefNum) 
 
   B1D36: 3D47 0014      2000014 mnq_6    MOVE D7,funRslt(A6)
 
   B1D3A: 4CEE 1CE0 FFE6 200FFE6 mnq_7    
MOVEM.L q_1(A6),D5-D7/A2-A4
   B1D40: 4E5E           'N^'             UNLK    A6
   B1D42: 205F           ' _'             POP.L   A0
   B1D44: 4FEF 000C      'O...'           LEA     12(A7),A7
   B1D48: 4ED0           'N.'             JMP     (A0)
  

Referring to the procedure I named open_Snd_Prefs, it sets up a Link frame and calls _GetResource to get the value of a 'STR ' resource with id = BF48. I rummaged around the System file in Resorcerer and found that this string had the value of “Sound Prefs”. The code then checks to see that it gets the string, and the rest of the procedure uses the Alias Mgr, etc to attempt to find its prefs file somewhere in the system folder. Apparently this procedure exits with 0 if it has found and opened the prefs file or a non-zero return when it has not. The caller futzes around to get the value of some magic resource from it and then closes the prefs file via _CloseResFile. So how, you may ask, could anything go wrong with this simple code?

Well, my debugger disconnects itself from the System file. That is, inside my Debugger, the System file is not on the chain of resource files to be searched, and, in almost all cases, it contains copies of the necessary resources that would normally come from the System file. The reason for this is that an open resource file contains a field for each resource that points to the copy of it in memory when the resource is opened. In order to avoid having to switch the values of these fields when transitioning between The Debugger and user, I chose the time efficient method of duplicating the necessary resources inside my debugger.

With this fact in hand, lets look at open_Snd_Prefs again. In C the source code would look like:

OSErr open_Snd_Prefs() {
  handle h = GetResource('STR ',$BF48); // get name of Sound Prefs file
  if( ! h ) 
    return( ResError ); // assumes the Resource Mgr returns a non-zero error
  {
          // locate and Open sound prefs file, ...

          return( 0 )
  } 
}

As I mentioned at the beginning, the resource manager may return a NIL handle, but it rarely returns a non-zero value of ResError.

The upshot of all this is that the caller got a 0 error return and thought that the sound prefs file was open. When it did the Get1NamedResource, and it failed, it then tried to close the current resource file, which in this case was my Debugger, and things went to hell from there.

In retrospect, a more robust way to specify and code the routine might be to have it return the (FCB) refNum of the sound prefs file that it opened or 0. Then the logic in the calling proc would be somewhat cleaner and less subject to failure.

Rewriting, the revised code would be:

int open_Snd_Prefs() {
  int refNum = 0;
  handle h = GetResource('STR ',$BF48); // get name of Sound Prefs file
  if( h ) 
  {
      // use Alias Mgr, etc to locate and open Sound Prefs resource file
      refNum = ??    // some value returned by HighLvlFSDisptch 
  }
  return( refNum ) }

As a last thought, I suspect that the buggy version of the code has been distributed by DTS as sample code for you to use, and in my humble opinion, it leaves something to be desired. Not because it is intrinsically wrong, but because the Macintosh Resource Manager is inconsistent about returning a non-zero value of ResError.

Some Things That You Can Try At Home:

1) Search your code for ResError calls and convince yourself that they serve some purpose, or that you won’t do anything really stupid if it returns 0 when it should not.

2) If you are using a class library such as MacApp, Metrowerks PowerPlant or TCL, then search it for ResError calls.

3) Selectively remove resources from your product and see what kind of stupid things it does.

4) Selectively remove resources from some one else’s product and see what kind of stupid things it does. Submit bug reports.

An alternative title for this article could have been “ResError Considered Harmful”. During the language wars of the 60’s and 70’s, I believe that Dijkstra threw the first stone by writing an article in SIGPlan, the ACM’s Special Interest Group of Programming Languages titled “Goto Considered Harmful”. His thesis was that the use of the goto statement made for poor programming style, etc. Over the next few years, the wars escalated to the point where I suspect that someone wrote an article titled “Harmful Considered Harmful”.

[We welcome your comments, feedback, and debugging tales of woe and intrigue at editorial@xplain.com - Ed stb]