SPM, Be My Host!

Considerations on plugging a compiler into the Symantec Project Manager

By Drew Sarkisian and Farooq Butt

Introduction

The majority of developers for Mac OS-compatibles employ some form of Integrated Development Environment (IDE) for application development. The two most popular IDEs available for the Mac OS are Symantec’s Symantec Project Manager (SPM) and Metrowerks’ CodeWarrior environments. These IDEs provide intuitive interfaces, simplified project management, extremely quick compilers and linkers, GUI development frameworks and integrated debugging facilities for fast application development.

Motorola has offered highly optimizing C, C++ and FORTRAN compilers for the Mac OS since May 1995. These compilers are hosted in the Macintosh Programmer’s Workshop (MPW) environment, a UNIX-like command line-driven shell. Our compilers have had a mixed reception: Mac OS developers are very enthusiastic about the quality of the code generated, yet most developers would rather work under either the SPM or CodeWarrior environments. We have received numerous requests to rehost our compilers to SPM and CodeWarrior.

This article discusses the story of our first IDE port, to SPM. We will try to give the reader a flavor of our experiences: the quick victories, the challenges, and last but not least, the problems. We hope this article will give tools developers a better understanding of the issues involved in creating a compiler plug-in of this kind. Please note the following:

• This article covers only our C compiler (mcc). The port of our C++ is in progress.

• The SPM-hosted compiler is hosted only on PowerPC. We don’t have any information or advice for folks who would like to create 68k (or FAT) tools to run under SPM.

Background Of MCC

mcc is an ANSI-compliant C compiler that generates highly optimized code for the PowerPC architecture. The compiler was initially designed to run on an AIX PowerPC host and generate code for an AIX PowerPC target (IBM XCOFF object file format). mcc has been successfully ported to the Parsytec PARIX PowerPC platform (host/target), SunOS/Solaris SPARC platforms (as an AIX-targeted cross compiler), Windows NT for PowerPC (host and target), and even MS-DOS (running as an extended-DOS application). The second Developer’s Release (DR2) of our MPW-hosted compilers was released in December of 1995.

These compilers (we claim) currently generate the most efficient code of any Mac OS-hosted PowerPC compilers available to date. The major benefit gained from using the Motorola compilers is the speed of the final executable. The technology comes from a UNIX background, where compilers are generally judged on the efficiency of the code they generate. The Macintosh platform has for a decade been a model of ease-of-use, user-oriented computing; this is reflected by the speed and elegance of the SPM environment. By porting mcc into SPM, we’ve opened the door for SPM developers to take advantage of our compiler technology while retaining the use of their favorite IDE.

There are drawbacks to using mcc: the compiler’s memory requirements are much larger than those of the native SPM compiler, and the speed of mcc itself is much slower. mcc was never designed to be a “quick” compiler; its claim to fame is the quality of the generated PowerPC code. The relatively slow compile speed, though, shouldn’t be a deterrent from using mcc under SPM. The scenario we envision typical developers following goes something like this. The native SPM compiler (Symantec C/C++) would be used for day-to-day development (fast compile/debug cycles), while mcc would be used to rebuild the product at reasonable intervals, (1) to ensure C source compatibility, (2) to make certain that such coding errors weren’t introduced as might trip up an optimizing compiler, and (3) to see just how fast the application runs.

Symantec, for its part, has provided an open interface to the SPM (which is what this article is really about). Symantec recently announced the availability of Apple’s MrC/C++ compilers as SPM drop-ins. The work was done by Symantec engineers, which may erroneously lead some developers to believe that only Symantec can rehost translators as SPM drop-ins. As we show in this article, this is certainly not the case! Nearly all of the work that was done hosting mcc as an SPM drop-in was done at Motorola in Austin, TX. Given that their drop-in interface was fairly new, Symantec proved very supportive and provided technical help to speed up the process. The majority of our drop-in was created without Symantec’s direct involvement. Taking advantage of many of the features of the SPM environment does not have to be an “inside job.” As Symantec refines their drop-in architecture, we expect future drop-in ports will not require any aid from Symantec.

[Such refinement is constantly under way, and anyone working on a translator should contact support@devtools. symantec.com to be certain of using the latest specs. The v8r5 release, though, has virtually no changes to the spec, we are told. In other words, the message from Symantec is, don’t allow concerns about the continuing evolution of the architecture to prevent you proceeding. - man]

The Important Issues

In the most generic sense, compilers that are driven from the command-line have fundamental differences from those driven from (Macintosh) IDEs. Here we’ll bring up some obvious (and maybe some not so obvious) points of difference.

Command-line compilers take their options and arguments from a command line typed by a user or generated by a Makefile. On the other hand, compilers hosted within IDEs have visual interfaces via dialog boxes, preference panels, etc. Of course adding these user interfaces to a product isn’t rocket science (unless you’ve never written any GUI code on a Mac - then, it can be like getting cold fusion up and running!).

A much more serious difference concerns the allocation and freeing of system resources. Many compilers from UNIX (or even Mac’s MPW environment) don’t do a good job of cleaning up all of the heap memory they may allocate, since the system (or MPW shell) will reclaim that space automatically when the compiler process terminates. Keeping this memory allocated when running under an IDE would create a memory leak, eventually crashing your IDE or the Mac itself. A similar situation arises concerning leaving open file descriptors lying around. Simply stated: command line-based compilation systems often assume that each executable component will be launched by a shell and that when its particular job is done, it will be cleaned up after.

A related assumption made by command-line compilers concerns the initialization of static data. Upon each invocation of a command-line compiler, the system loader will initialize all of the compiler’s static variables to either 0 or pre-set values determined at compile time. A compiler hosted under an IDE may need to remain resident, and might have to explicitly re-initialize its own static data between compiles.

Compilers from non-Macintosh environments often use ANSI C file I/O functions (fopen(), fread(), etc.) to process source files for compilation, producing temporary files, and generating resultant object files. Such compilers deal exclusively with ANSI file I/O abstractions such as pointers to files, etc. This may sound alien to folks used to Macintosh Handles or FSSpecs, but it’s a fact of life in the world outside Mac OS. While file I/O is essentially file I/O, discovering that a stat() call under ANSI file I/O translates to a FSMakeFSSpec(), followed by a ResolveAliasFile(), followed by a PBGetCatInfo() is not obvious to tools developers porting their software to the Mac OS.

IDEs have their own ideas on how to locate source files and headers files. The most recent copy of a file could be open in an IDE editor window. When the user asks for that file to be compiled, an IDE-hosted compiler has to know that the file in question is in memory, and not on disk, and access it accordingly.

When a programmer presents some syntactically incorrect program to a command-line compiler, the compiler presents the error messages directly to the terminal (console, window) that processes the command-line. This is not very useful when running under an IDE since there is no “console”; the compiler, therefore, needs a mechanism for communicating errors to the IDE in such a way that the IDE can present the errors using its errors/warnings window.

Something often overlooked when porting software over from UNIX to the Macintosh is the fact that the Mac OS is a cooperative multitasking environment. This means that you must find a way to share processing time between your drop-in and the IDE it is running in. If you don’t, the users will have no way of interrupting a long build, or performing other tasks while a compile is taking place. This facet of the Mac OS is an interesting challenge when initially porting a compiler.

Symantec Project Manager 8.0

SPM (version 8.0 and higher) is an open environment; with some amount of effort, developers can port their own development tools to run under the SPM. These ported tools are called “drop-ins”; with the 8.0.4 release of SPM, Symantec provides all of their compilation tools as drop-ins.

Languages currently supported by SPM include C, C++ and Pascal: the IDE provides a syntax-highlighting editor for these languages. Work is in progress to support FORTRAN in the near future.

Supported object file formats is a sticky question. The standard object file format for PowerPC is XCOFF, inherited from IBM’s AIX. While SPM projects can process XCOFF library files, the internal object file format of the project is a proprietary invention of Symantec. Current development efforts favor drop-ins that generate XCOFF object files, rather than arbitrary object file formats.

SPM offers a choice of two linkers: the SPM (internal) incremental linker, or MPW’s PPCLink (invoked from within SPM via ToolServer). The incremental linker is very fast, and allows for debugging of SPM-built applications, but it normally supports only Symantec’s internal object file format (this restriction is currently being addressed - more on this later). The PPCLink interface allows drop-ins that generate XCOFF object files to be used along with the SPM, but sacrifices the ability to use the Symantec debugger on resulting applications.

Compiler Toolkit 2.1

Symantec provides a “Compiler Toolkit” with the latest versions of SPM. This toolkit is designed to give the aspiring tool developer the basic information needed to convert his/her compiler or translator into an SPM drop-in. It includes documentation (some of it rather out-of-date), an example project, and pre-built libraries (some with source!) needed for creating a drop-in translator of your own.

The first place to start is looking over the included documentation (in Adobe Acrobat format) found in the Compiler Toolkit 2.1:documentation folder:

• Compiler Writer's toolkit.pdf

• SPM Compiler Writers guide.pdf

• SPM Debug format.pdf

• SPM Options.pdf

Compiler Writer's toolkit.pdf is the most immediately useful document. It is an overview of the entire Compiler Toolkit, and includes a detailed walk-through for porting the empty translator example (source code and projects provided, of course). This document also contains information on whom to contact at Symantec for more information and/or assistance. The remainder of this article will refer to this document. The next two documents were not used during our initial project port; they contain some interesting information, but are not currently up-to-date. Finally, SPM Options.pdf covers how to set up options dialog boxes for your translator to run under SPM.

While Symantec recommends that you build your drop-in under SPM, they went to the trouble of providing all the interfaces and libraries necessary for building your drop-in under MPW as well. We appreciated this very much, since we built the current MPW-hosted mcc compiler with mcc under MPW. Having the ability to re-use our MPW makefiles for the SPM re-host project greatly increased our productivity, since we didn’t need to invest any time re-inventing (and debugging) build procedures. The MPW-specific portions are in Compiler Toolkit 2.1:(MPW).

There are numerous project files in the Compiler Toolkit 2.1 directory that all refer to the example “empty” translator drop-in. The two most immediately useful to us were:

• PPCempty.Π Symantec Project Manager project for a PowerPC host.

• empty.make MPW makefile for the empty translator.

The example proved to be quite helpful. It gives a simplified overview of the minimal requests that a drop-in needs to support in order to work correctly in the SPM environment.

The most pleasant surprise in the entire package was the SPM drop-in support library. The entire package has been well organized and provides excellent abstractions to ease the task of interfacing your drop-in with SPM. The folder Compiler Toolkit 2.1:SPM_Libraries contains the following files and folders of interest:

• PPCSPMLib.o (PowerPC) SPM callback interfaces.

• PPCSPMThinkxcoff.o (PowerPC) XCOFF object file manipulation
routines.

• SPMShell.r Resource file for building a drop-in
with a minimal options interface.

• :thinkxcoff: Thinkxcoff library sources.

• :spmlib: SPMLib sources.

Having the source files available for the SPM support routines was especially convenient. Not only did we learn a great deal about how these interfaces were actually supported, we also managed to fix a minor bug in the library routine __SPM_error().

Finally, the folder Sym C++ for Power Macintosh:Standard Libraries:PPC Shared Libraries: contains the PPCANSI and PPCUnix shared libraries. The first is necessary, while the second provides some simple support for UNIX-style functions such as getpid(), etc.

Restrictions on Drop-in Tools

It’s difficult to make a complex environment such as SPM be both open and efficient; yet Symantec has managed to do a good job at both. Still, you can’t have everything; there are certain limitations and restrictions you’ll have to deal with to host your compiler under SPM.

• Never call the ANSI exit(), assert() or abort() routines from your drop-in! This is a quick way to blow away the SPM IDE itself, rather than merely terminating the execution of your drop-in.

• Do not allow any use of the ANSI console I/O. Although console calls such as printf() and scanf() are supported by the Symantec-provided PPCANSI library, using ANSI console I/O (i.e. any routines that use stdin, stdout or stderr) wreaks havoc with SPM’s window management.

• SPM does not currently support multi-process drop-ins. This pertains to some compilers that consist of two or more separate executables that are called in sequence to perform different stages of the compilation process.

• You must use Symantec’s PPCANSI libraries. Why? Because the library of callbacks into the SPM (SPMlib) relies on it. While this is not a major problem on the surface, it does give you a chance to make an interesting and frustrating mistake. If the compiler you are using to compile your compiler (yes, we did say that!) translates "\n" to "\r" (and vice-versa), your final application will not function correctly with the PPCANSI library routines. For Symantec compiler users, you must use the -nomapcr option to suppress this translation when compiling your drop-in.

Communications Between SPM And Drop-Ins

At this point, we’ll be covering the initial interfaces between drop-ins and SPM. This part isn’t difficult to conceptualize, but it’s nice having it spelled out in obvious terms.

SPM expects to be able to give the following requests to any drop-in compiler:

reqInitialize Drop-in is to perform any
needed initialization.

reqCleanup Drop-in is to clean up any
resources it may still have
allocated.

reqCompile Full compile.

Other requests that you may wish to support include:

reqCompileDebug Full compile and produce
debugging information.

reqCheckSyntax Check source for syntax errors
only.

reqPreprocess Run the C preprocessor and
return the preprocessed result.

reqPrecompile Produce a precompiled header
file.

reqDisassemble Produce an assembly listing
(only) of the file.

These requests are defined in Compiler Toolkit 2.1:(MPW):Interfaces: SPMIncludes folder. How does SPM communicate these requests over to the drop-in? Your drop-in must have a main() entrypoint which accepts the following arguments:

 main (long int placeholder, struct Intf *pintfLocal)

The first argument’s name speaks for itself. The second argument is a pointer to the SPM options block. This data structure contains all the information concerning the particular action that SPM is requesting. This includes the specific request, the name of the file to be compiled, a pointer to the options to be used for this particular compile, etc.

One of the easier modifications to make is to create a new main() entrypoint for a drop-in and to simply rename the original main() entry orig_main() (or something a little more creative). With a little preparation, you can call orig_main() from main(), cleanly separating the SPM-specific processing from the common (and generic) processing your compiler originally performed. An example at this point seems appropriate:

Listing 1: Example drop-in main() entrypoint for a prospective drop-in

 int main (long placeholder, struct Intf *pintf)
 {
 int argc;// Arg.  count
 char **argv;    // Arg.  vector: pointer to array of strings
 char args[512]; // Local copy of arguments
 struct Options *poptions; // Pointer to cmd-line
 int success;
 
    // Make a global copy of pintf for use throughout our drop-in. A matter of taste.
 global_pintf = pintf;
 
    // Allow SPM to initialize the data in the Intf structure.
 __SPM_init (global_pintf, 0);

    // Handle the particular request from the SPM.
 switch (global_pintf->request) {
 case reqInitialize:
 case reqCleanup:
 success = 1;  // SPM wants 1 for success!
 break;
    // Treat the following two the same, but you don’t have to.
 case reqCompile:
 case reqCompileDebug:
    // Get at the options for this compile. For this example, they are provided as 
    // a string, a la a command line! Very convenient!
 poptions = *((struct Options **)SPM_pintf->options;
 
    // __SPM_cmdparse() breaks up a string as if it were a command line, and fills 
    // in the typical argv array of strings. It returns the number of arguments.
    // Note that we are providing the name of the program that was “called” as 
    // the first argument. Remember: if this were called from a command-line, 
    // argv[0] would be pointing to the name of the compiler executable!
    // This helps us avoid any special-processing in the rest of the drop-in source.
 strcpy (args, "my_dropin_name_goes_here");
 if (strlen(poptions->prefix))
 strcat (args, poptions->prefix);
 
 argc = __SPM_cmdparse (args, &argv);
    
    // Since our original “main()” entrypoint returned 0 for a successful compile, 
    // reverse the sense of its return value (SPM wants a 1 for success).
 success = !orig_main (argc, argv);
 break;
 } /* end switch */

// If there was a problem completing the request, let somebody know. Depending on 
// your circumstances, you may wish to get more specific here.
 if (success == 0)
 SPM_pintf->shortMsg = "Something went wrong.";
 
// Clean up after ourselves.
 __SPM_term (global_pintf, 1);
 return (success);
}

This example is a bit simplified, and is very much like the example provided in the Toolkit documentation. The point is that this code provides a good starting point for any drop-in compiler port. Let’s go over some of the more important aspects of the example; then we’ll move on to what we specifically had to do for the mcc drop-in.

Options Processing

Creating dialog boxes for options processing is probably not the most critical portion of any drop-in. You will have to do it sooner or later, but later may be a better choice if you don’t want this task interfering with your initial porting efforts.

The example illustrates Symantec’s solution to the problem. In order to get you up and running as soon as possible, they provide you with the following C code, resources, and library routines to accept a string that can be treated like a command-line:

• SPMshell.r - This contains all the basic resources ('DITL', 'STR#', 'INFO', 'cfrg', etc.) you’ll need to get your drop-in up and running. It also includes resources that gives you a “prefix” area to type in command-line options for your drop-in.

• emptyoptions.h - The C source code needed to associate the 'CNFG' resources with the struct Options used in the example.

If your command-line compiler originally used the argc/argv mechanism for processing command-line arguments, you can get a “command-line” from the prefix built into your drop-in (set in Options : <Name-Of-Drop-In>). Use the __SPM_cmdparse() library routine to break it up into an array of strings (argv) and to get a count of the number of arguments (argc).

This solution has worked out so well, we’ve still managed to put off the task of creating a real dialog box interface!

System Resources: Heap Memory

Symantec took pity on those potential drop-in translators that do a poor job of freeing up heap memory which they managed to allocate during a compile. If your drop-in uses standard ANSI library function calls to allocate heap memory (malloc(), calloc(), realloc()), Symantec provides two means for automatically freeing up all heap memory that your drop-in allocates.

The first method is demonstrated in the example code. The call to __SPM_term() is the key:

 __SPM_term (global_pintf, 1);

The first argument is the SPM options block; nothing new here. The second argument is a flag telling __SPM_term() to free up all heap memory allocated by the drop-in.

The second method is a bit more explicit. Symantec’s PPCANSI library contains a routine called __malloc_cleanup(), which will free up all heap memory allocated up to that point by the current executable/shared library. If the organization of your drop-in allows you to utilize this routine, use it. Needless to say (but we’re saying it anyway!), be extremely cautious when and how you use this routine.

System Resources: Global Variables

This wasn’t nearly the problem we expected it to be. For PowerPC native drop-ins, SPM utilizes the Apple Code Fragment Manager to re-establish a connection to the drop-in translator each time SPM invokes the drop-in. The net effect is that the drop-ins global variables are re-initialized and ready to go. While this does introduce a reloading penalty, in practice this has not proven to be significant.

Enhancing the SPM with Your Drop-in

“How does one present a new drop-in compiler to the SPM?”, you ask. The SPM looks for a folder called (Translators), in the same folder as the SPM itself. Any file of type 'TRAN' that SPM finds in the (Translators) folder is considered an available drop-in. SPM identifies translators internally by their creator ID, not by name; you must choose a unique creator ID for the SPM to distinguish between translators.

Once this is done, SPM (upon startup) will include the new drop-in in the list of translators that can be utilized for your project. Choose the Options menu item and go under Extensions to associate your drop-in with a particular file extension (.c, .s, .xcoff, etc.). You will also find under the Options menu a specific options panel for your new drop-in, identified by name.

Sounds quite simple: make a few quick modifications and voila! You have a brand-new translator drop-in for the SPM environment! Well, here are some of the things you would need to know, gathered from our experience porting mcc into the SPM world.

Specifics On Porting MCC

“A nice overview”, you might say, “but not enough information. Heck, you haven’t even covered how to deal with C source files, how to pass back object files to the SPM, how to report errors ” All this is true. Now that we’ve clouded your minds with pages of terminology and issues, we’ll get down to the nitty-gritty of what you need to really get the job done.

mcc Organization

mcc is typical of many UNIX compilers in that the executable “mcc” is a driver program that processes command-line arguments and calls other programs to compile your C program. This is true on every platform we’ve ported the compiler to, including MPW. Motorola’s compiler consists of five separate executables:

mcc - Driver program.

cfe - C front end.

ipa - Interprocedural analyzer.

cor - PowerPC code generator; generates assembly files.

pas - Motorola Portable Assembler; generates XCOFF object files.

Unfortunately, SPM doesn’t support multi-executable drop-ins. We couldn’t simply create a single large executable that did all this work (well, we could have, but the changes to the base compiler source would not have been easy). In the initial phases, this appeared to be a major problem. However, a simple solution soon presented itself: make mcc the drop-in for SPM, and turn the other compiler phases into shared libraries, each with a single exported entrypoint. This had the advantage of keeping all the global variables in each phase separate (there were five versions of curline and index).

The SPM-specific portion of the mcc driver looks very much like the example presented earlier, with the exception that the old_main() portion (read: the original mcc driver) now calls library routines cfe_main(), ipa_main(), etc., rather than spawning off new processes to perform the compilation. Each of these library routine entry points still uses the argc/argv style of argument processing.

A bit of advice: if you can, start with a small portion of your prospective drop-in and rehost it under SPM. In our case, we took our assembler, pas, and created a new drop-in assembler that translates Motorola PowerPC assembly language files. This drop-in assembler generated standard XCOFF objects, and helped us work out some basic issues while tackling a relatively small porting job. It was worth the effort; we managed to keep over 95% of the work we did on pas by the time we completed the entire mcc port.

File Management

mcc deals with three types of file when performing a compilation: source files, temporary files, and object files. SPMlib provides abstractions for dealing with all three types of files that fit in very well with the overall mcc source code.

1 Finding Source Files

SPM provides an easy-to-use abstraction for finding the source files to be compiled, whether they are on disk or in memory. If you know the name of the file to open, SPM will handle the rest.

SPMLib provides the following callback routine:

 
 FILE *
 __SPM_opentextfile(
 struct Intf *pi,// Pointer to SPM options block.
 char *filename, // File to open (C-string).
 int sys_incl,   // Non-zero if filename was part
    // of #include statement using
    // ‘<>’; 0 if ‘“”’ were used.
 int *SPM_fileno,// SPM internal file number.
 short *SPM_error);// Non-zero in the event of
    // any errors opening filename.

Upon successful completion, __SPM_opentextfile() returns a FILE * to the open file. The drop-in needs to keep track of SPM_fileno if it needs to report any errors or warnings against the particular source file.

A related routine, __SPM_opentextfilereplace(), is functionally equivalent to __SPM_opentextfile() but converts all "\n"s to "\r"s and vice versa.

This interface reduced our source file management chores to a minimum. mcc no longer needed to keep track of search paths for sources, nor did it need to get involved with any source-file caching schemes. Both of these jobs are best left to SPM itself.

2 Temporary Files

Like many other compilers, mcc uses temporary files during the compilation process. In the UNIX environment, these files reside on disk, typically under the directory /tmp, and exist only while the compiler is running. When mcc finishes, it cleans up all temporary files associated with that particular compilation.

Using temporary files under the SPM environment appeared to be more problematic. One issue was where to put these temporary files. We were not aware of any standard folder that could be used by applications for temporary files. Another not-so-obvious issue was a bug in the SPM which manifested itself when using ANSI file I/O for disk files. Back in August 1995, we had managed to get a primitive demo version of mcc running under SPM. This compiler performed no optimizations, and could barely compile our demo SillyBalls (a bouncing ball demo). Still, it was progress, and showed that we could host mcc under SPM. This version of the compiler used disk temporary files residing under the Sym C++ for PowerPC folder. The compiler manipulated these temporary files just fine - opening, reading, writing, and closing the files was not a problem. Unfortunately, after running the compiler, we were not able to shut down the SPM; if we did manage to do so, the machine would crash!

While neither Symantec nor Motorola had any idea as to why this was happening, it became a moot point. Symantec developed SPMlib callbacks for creating memory-resident temporary files. This not only got around the bug, but improved overall compiler performance by avoiding disk accesses.

The new SPMlib interfaces were:

 
 FILE * __SPM_opentempfile (void);
 Handle __SPM_closetempfile (FILE *);
 FILE * __SPM_openhandle (struct Intf *, Handle);

__SPM_opentempfile() takes no arguments. It returns a standard ANSI FILE * which works correctly with Symantec’s PPCANSI library.

__SPM_closetempfile() accepts a FILE * to the file to be closed and returns a handle to the memory that the temporary file occupied. In this way, you can actually keep the contents of a temporary file around, if necessary, and re-examine the contents at any time.

__SPM_openhandle() accepts a pointer to the Intf structure your drop-in currently is using, and a Handle you wish to open as a FILE *. A limitation to this routine that is worth remembering: you should only be opening a Handle as a FILE * for read-only! You cannot append to a FILE * that was created using __SPM_openhandle().

These three routines allowed us to create a generic abstraction that replaced the normal fopen(), fclose(), fflush() and unlink() calls with ones that utilized the SPMlib interfaces. This allowed us to utilize memory-resident files without having to modify the vast majority of mcc.

3 Object Files

Thus far, we’ve been able to open text files for compilation, and manipulate temporary files for use during compilation. How do we manage to hand object files back to SPM?

The SPM IDE expects all final XCOFF objects to be in Handles. After your compiler is all done, if you were using ANSI I/O, you’ll have a FILE * to an object file. If you were using Mac Toolbox I/O, presumably you’ve already called FSRead() to copy the file into a Handle and you can skip the next step. If you opened the object file using __SPM_opentempfile(), the result of closing the object file via __SPM_closetempfile() is a Handle. The next step is to take the Handle to the XCOFF object and process it using:

 PTHINKXCOFF txc_thaw (Handle);

txc_thaw() takes a Handle that contains a standard XCOFF object file and returns a pointer to a THINKXCOFF data-structure (Symantec’s internal representation of XCOFF). If there was something wrong with the object data in the Handle, txc_thaw() returns NULL. This routine is part of Symantec’s thinkxcoff library.

SPM provides a general-purpose callback for drop-ins to send the results of an SPM request back to the SPM itself:

 (void) _SPM_SendResult (struct Intf *,
 long int,// Result type:’XCO1’,etc.)
 void *); // Pointer to Handle
    // containing XCOFF object.

This routine can minimally accept result types of 'TEXT' (typically the result of a reqPreprocess request), 'XCO1' (XCOFF object generated using txc_thaw()), and 'XCOF' (XCOFF object file libraries). Other result types may be possible; we have no experience with them.

A related issue concerns the ability to utilize the SPM internal / incremental linker with mcc-produced XCOFF object files. Symantec originally developed their own internal object file format for handling PowerPC code; the SPM internal linker was limited to using this internal format. Symantec has enhanced the latest pre-release version of SPM to handle mcc-produced XCOFF object files. Work-in-progress includes supporting mcc-produced debugging information for use with the Symantec debugger. This will provide mcc with full integration within the SPM environment.

Compiler Error and Warning Messages

Most compilers have one or two central routines that deal with displaying error and warning messages to the user. These routines display the file name and line number where the error was found, display the error (or warning) message, and optionally display a portion of the offending source (context). Command-line compilers send the warnings and errors to the console (stderr) using fprintf() calls.

This is, of course, not done under the SPM environment. SPM provides a library routine with which to present errors to the IDE:

 
 (void) __SPM_error
 (struct Intf *pintf,// You’ve seen this before!
 char *perrmsg,  // C-string; body of error message
 int warning,    // kError, kWarning or kDebug.
 int lineno,// Line number of offending source.
 int fileno);    // SPM internal file number of
    // offending source.

The string perror may include multiple lines separated with the 0xD character (\r)). The warning parameter derives its value from the __SPMErrorType enumeration, declared in SPMlib.h. The fileno parameter was obtained via the call the drop-in made to __SPM_opentextfile() or __SPM_opentextfilereplace().

Cooperative Multi-Tasking

Users may wish to do other work while some application is compiling in the background in SPM. At the very least, your drop-in needs a way to share processor cycles with SPM, as well as allow the user to interrupt a compilation in progress.

 
 int __SPM_ReportProgress
 (structIntf * pintf,
 long   num_lines);// Total # of lines
    // currently compiled.

__SPM_ReportProgress() should be called many times from your drop-in. If the drop-in you are porting is fortunate enough to be coming from the MPW environment, the hard work of identifying appropriate places to call __SPM_ReportProgress() has been done for you. In general, you can simply replace calls to the MPW routine SpinCursor() with calls to __SPM_ReportProgress().

How many times to call __SPM_ReportProgress() is an interesting issue. Too many calls will drag down overall drop-in performance, while too few calls will give the user sluggish response from the IDE. The following code fragment roughly represents the code that mcc uses to resolve this issue.

Listing 2: Interface to call __SPM_ReportProgress()

#define HZ 60

void
mcc_multi_task (struct Intf *SPM_pintf, int total_lines)
{
 static unsigned long last_time;
 static int num_times = 0;
    /* Each “tick” is about 1/60 of a second. */
 unsigned long now = TickCount();
 int num_to_report;

 if (now - last_time > HZ/10) {
 last_time = now;

    /* If we are the C front end, give the number of source lines that have been 
    processed thus far. If we aren’t (i.e. we’re in the code generator), report a
    “heart-beat” to let the user know that the compilation is still taking place.
    */
#ifdef C_FRONT_END
 num_to_report = total_lines;
#else
 num_to_report = num_times++;
#endif

    /* If the user has interrupted the compilation process, call mcc’s local exit.
    This is NOT a call to the real exit() routine! */
 if (__SPM_ReportProgress(SPM_pintf, num_to_report) != 0)
 {mcc_local_exit (1)};
}

We had previously solved this problem when mcc was hosted under MPW; the call to __SPM_ReportProgress() was a call to SpinCursor(). This call never appears directly in the mcc source; a macro is used instead to allow for easy exclusion of the call for host platforms that don’t require this sort of routine.

It’s best to experiment a bit with the number of times you call __SPM_ReportProgess(). We found that for mcc, a good number was about 10 times per second. This allows for reasonable system response while not throttling the compilation in progress.

One small sideline: mcc compiles source differently from the Symantec compiler. Symantec will compile C source code one function at a time; when SPM reports that it has compiled 1000 source lines, you can count on that figure. mcc, on the other hand, will go over the entire C source file with the C front-end first, generating an intermediate representation that is later processed by the optimizer and the code generator. Since code generation takes up the majority of processing time for mcc, it appears to users that the compiler has “frozen” under SPM when the line number count stops incrementing.

The conditionally compiled code in mcc_multi_task() is the majority of the solution. Calls to mcc_multi_task() from mcc’s front end pass the appropriate line number along for display. When the front end finishes, it saves off the total number of lines processed. Calls from the code generator then display a sequential count, rather than line numbers, to indicate code generation progress. When the compilation process is done, the mcc driver makes a final direct call to __SPM_ReportProgress() with the final source line count; this is the value that SPM adds to the Total Lines field of its (compilation) Progress window.

Conclusions

Symantec has provided a robust set of callbacks to support non-Symantec tools under SPM. The entire process of porting mcc into the SPM environment was a rewarding one. In retrospect, it wasn’t difficult, and was certainly worth the effort. Experienced Mac OS developers should have an easy time dealing with the SPM porting requirements. We certainly encourage tools developers to port their software to run under SPM. Symantec provides an excellent user interface, and there is no need to “reinvent the wheel”.

Credits

Firstly, we would like to thank John Micco, Mark Romano, Tom Cardoza and the rest of the Symantec crew in Bedford, MA, for their patience and tireless assistance. These folks have been a joy to work with.

On the Motorola side, we’d like to thank Julie Shipnes-Allen and Tom Wood (for their continued support of the Macintosh platform), the entire Motorola RISC Software Compilers and Tools team (for constantly improving mcc and our other development tools), and Mike Phillip (manager of the Compiler and Tools Group, who brought us all together years ago). Finally, a special thanks to Howard Thamm, whose tireless efforts ensured that this article was actually completed.

The PowerPC name and the PowerPC logotype are trademarks of IBM Corp., and are used under license therefrom. All other trademarks and registered trademarks are the property of their respective owners.

[Between the time this article was written and when it went to press, Motorola was forging ahead with version 3.0 of their C/C++ SDK. This, as the name implies, includes compilers for both C and C++, which run not only as MPW tools but also as plug-ins for CodeWarrior as well as for SPM. - man]