Volume Number: 15 (1999)
Issue Number: 4
Column Tag: Plugging In

How to Write a Photoshop Plug-In, Part 1

by Kas Thomas

How to harness the awesome power of Adobe's 800-lb graphics gorilla

Introduction

Over the years, Adobe Photoshop has become such a staple of desktop graphics that people tend to lose sight of what may well be this program's most significant contribution to the world of desktop computing: namely, the popularization of plug-ins. External code resources were nothing new in Photoshop's early days, of course; Silicon Beach already had a plug-in service in SuperPaint, for example (and before that, HyperCard developers championed XCMDs). But Photoshop was the first bonafide "killer app" to make extensive use of third-party plug-ins, leveraging the efforts of outside programmers to expand the host program's core functionality. It was a brilliant maneuver, ensuring Photoshop's hegemony in the graphics-software market. Thanks to Photoshop, user expectations have been raised to the point where, today, no developer of a major software title - of any kind - can afford to neglect to provide a plug-in architecture. It's hard to find a compiler, web browser, data-compression utility, page-layout program, video editing suite, or 3D modelling or animation program that doesn't rely on a plug-in architecture for much (if not most) of its functionality.

The graphics programmer benefits tremendously from a plug-in mechanism, for the simple reason that more time can be spent implementing image-manipulation code while less time is wasted on event-loop code, file I/O, format conversion issues, and other extraneous matters. If you're interested in simply trying out some new graphics-programming ideas, it's a lot easier to write and compile a small plug-in than it is to whip up a standalone test program. Consider these advantages of writing plug-ins:

• File I/O is handled by the host program.
• No format conversions are necessary; Photoshop will open almost any graphics file.
• No user-interface code. (Actually, you will probably want to have a parameter-setup dialog, although some plug-ins, like Blur and Blur More, don't have one.)
• Buffer pre-fetch is handled by the host program.
• Writing to the screen is handled by the host. No need to get involved with GWorlds, PixMaps, monitor modes, etc.; in fact, you'll seldom use Color Quickdraw.
• No need for error-handling related to file I/O, format conversions, buffer operations, blitting, or user interface code.
• "Undo" is handled by the host.

Another nice thing about writing Photoshop plug-ins is that most of them will work "as is" with other programs - for example, with Adobe PhotoDeluxe. Many non-Adobe programs also support Photoshop plug-ins. Thus, an image filter that you write for Photoshop may turn out to be quite useful to someone working with another product entirely.

Most of us think "image filter" when we think of Photoshop plug-ins, but the fact is, there's almost nothing you can't do inside a plug-in. For example, your plug-in code can make use of Quickdraw 3D, QuickTime, Open Transport, the Sound Manager, or any other code you'd normally access from a standalone program. In theory, there's no reason you couldn't implement a game, OCR program, or even a word processor as a Photoshop plug-in, if you wanted to. What you decide to do with all this power is up to you.

Getting Started

The first thing you'll need to do if you intend to write plug-ins for Photoshop is obtain the most current version of the Adobe Photoshop Software Developer's Kit (SDK). Generally, you can get everything you need from <http://www.adobe.com>, but if you want to obtain up-to-the-minute pre-release or beta versions of SDKs, you need to join the Adobe Developer's Association (ADA). Membership in ADA is a good idea for several reasons. First, you get the most current versions of all of the Adobe Graphics and Publishing SDKs (for Photoshop, Premiere, After Effects, PageMaker, and Illustrator) mailed to you on one CD, which is a lot more convenient than trying to download everything off the Web. The Photoshop SDK alone, for example, contains over 17 megabytes of sample code and around 10 megs of documentation in .pdf format. That's a lot to try to download.

Another reason to consider joining ADA is that you get developer technical support (i.e., answers to your development questions), and discounts on Adobe products. For example, as an ADA member, you can buy a full version of any Adobe product for $99. (There is a limit of one purchase per year per member, however; and you have to sign an affadavit affirming that your purchase will be used for development purposes only.) You have to admit, this makes the $199-per-year membership fee look much more attractive. Where else are you going to get a full version of After Effects for $99?

ADA members can also subscribe to the Graphics and Publishing SDK Discussion List and can receive Adobe Technical Notes automatically via e-mail. Visit <http://www.adobe.com/supportservice/devrelations/sdks.html> to learn more details.

Documentation

Version 5.0 of the Photoshop SDK comes with around 1,000 pages of .pdf documentation contained in a dozen files. Fortunately, you won't need to read most of this material, although you definitely will want to become well acquainted with the 168-page Photoshop API Guide Additional doc files talk about the TIFF and EPS formats, the all-new Adobe Dialog Manager, and the Plug-In Component Architecture API (otherwise known as PICA), which is a device-independent meta-API through which plug-ins can be invoked - and controlled - either by various hosts or by other plug-ins. Not only can your plug-in use PICA to pipeline its input or output through other plug-ins, but you can use PICA to implement daemon-like "service provider" modules (which Adobe calls "Suite PEA" plug-ins). The new Adobe Dialog Manager is implemented this way, for example. The idea behind ADM is to provide platform-independent support for dialogs, so that - as with Java - you just write generic dialog code once, and then it runs, works, looks, and smells the same on any host. (Newtek's Lightwave 3D - the popular cross-platform 3D tool - handles dialogs for plug-ins in exactly this fashion, via its Panel Services plug-in.) Thanks to PICA, you could very easily create custom math libraries, DSP transform engines, fractal generators, expression parsers, or any number of other sophisticated utilities as "Suite PEA" modules. The possibilities here are limitless.

In the future, all Adobe host programs (Illustrator, Photoshop, PhotoDeluxe, Premiere, After Effects, etc.) will go through PICA to find, load, invoke, and control plug-ins. If you want your plug-ins to be fully PICA aware, be sure to check out the PICA Guide.

Types of Plug-Ins

At this writing, Photoshop supports nine core types of plug-ins:

1. Image Filter. This is the most familiar kind of plug-in, since it operates directly on images to change their appearance. The familiar Noise and Blur plug-ins (and dozens of others) that ship with Photoshop are image filters.
2. Import. Prior to Photoshop 4.0, these were called Acquire modules. Import modules basically open an image in a new window. They can provide an interface to scanners or frame grabbers, read images in unsupported or compressed file formats, or generate synthetic images (e.g., gradients or fills), among other possibilities.
3. Export. There really is little need for this category of plug-in, since "Save As" operations involving format conversions can be dealt with via the Format plug-in type (below). The most appropriate use for Export plug-ins is to support unorthodox (non-disk) types of export operations, such as exporting data to a printer or other peripheral.
4. Parser. This is not a category you should worry about, since the interface is not public. Adobe uses it for importing and exporting Illustrator files.
5. Extension. You also won't be writing any of these, since the interface is not public. Adobe uses this plug-in type for certain low-level tasks like custom cursor-drawing and enabling asynchronous I/O.
6. Format. Plug-ins of this type are also called File Format or Image Format modules, since they provide support for reading and writing additional image formats. They show up as new options in the pop-up menu in "Open," "Save As," and "Save a Copy" dialogs.
7. Color Picker. This facility allows the user to substitute a custom color-picker (implemented as a plug-in) as part of the Preferences setup so that when the user pops the "Choose a Color" dialog, your custom color-picker appears instead of Apple's or Adobe's.
8. Selection. Modules of this type, which show up at the bottom of the "Select" menu in PS 5.0 or later, enable the plug-in writer to generate custom selections, paths, or layers on the fly. Good examples of this class of plug-in are given in Adobe's SDK. (No examples of this kind ship in the standard Photoshop retail distribution suite.)
9. Automation. This category, like the Selection category (above), is new with Photoshop version 5.0. It implements a very powerful class of plug-in based on Actions. Basically, any menu action or tool operation that a user can apply to an image (or to a layer, selection, or channel), including plug-in invocation, can be done from inside an Automation plug-in. The organizational paradigm underlying Actions is object-oriented, with many layers of abstraction, inheritance, etc. To harness the full power of Automation requires an understanding of OOP concepts, familiarity with the Actions scripting API, and a working knowledge of PICA (Adobe's meta-API for communicating between plug-ins; see above). Space doesn't permit a full discussion of Automation plug-in programming here. To do the subject justice would require a book-length treatment. Fortunately, such a book already exists: the 223-page Actions Guide in the Photoshop SDK docs.

Unless your primary need is to implement a new image format or compression scheme, chances are you'll be spending most of your time writing image filters, so that's what we'll talk about for the rest of this article. Adobe provides CodeWarrior Pro 1 and/or Pro 2 projects for 19 different plug-ins in the current SDK, covering all seven major categories for which public interfaces are available. Most of the projects are still implemented in C, although the newer ones (for example, the Automation plug-ins) are done in C++.

The SDK example projects, incidentally, contain a great deal of legacy code (for 680x0 processor Macs), with discussion of "fat" binaries, register A4 setup, etc. But since Photoshop 5.0 and up requires a PowerPC processor, it doesn't really make sense any more to compile plug-ins as anything but PPC-native code. If you're writing plug-ins for Photoshop 5.0 or higher, compile them as PPC-native. This will streamline your code, speed up compilation, give smaller executables, and make it possible for you to use global and static variables in your code (which you normally can't, in a 680x0 plug-in).

Plug-In Resources

When Photoshop launches, it scans plug-ins for 'PiPL' (plug-in property list) resources. This is how Photoshop knows what category your plug-in falls in (Filter, Automation, Import, Export, or whatever). In years gone by, this info was determined by the plug-in's filetype, but that's not how it works any more. If the PiPL resource says your plug-in is a filter, Photoshop will treat your plug-in as (and expect it to be) a filter, no matter what the filetype of the module on disk.

Prior to version 3.0 of Photoshop, plug-ins had 'PiMI' ("pimmy") resources to inform the host of various plug-in properties. The PiMI resource is no longer needed, however, unless you want your plug-in to work under Photoshop 2.0 or 2.5. Accordingly, it should be dropped in favor of the 'PiPL'.

The 'PiPL' resource has grown to be a sophisticated, flexible, extensible structure containing many property lists, so that plug-ins can fine-tune their relationship with various hosts (not only Photoshop but After Effects, Premiere, Illustrator, and others). For example, in a 'PiPL' you can inform Photoshop of the specific image types (RGB, CMYK, grayscale, etc.) your plug-in is designed to operate on, so that your plug-in's name will be grayed out in the Filters menu when the user is working in an inappropriate image format. You can also communicate preferences involving layer handling, transparency, buffering, and many other properties. To learn more, consult the Plug-In Resource Guide that comes with the SDK.

The SDK sample code projects contain templates for compiling PiPL resources. One thing you should be aware of is that no version of ResEdit (as of this writing) is PiPL-aware, so the best tool for graphic editing of PiPL resources remains Resorcerer. If you insist on using ResEdit, you can open the resource forks of the compiled plug-ins that come with the SDK, copy any PiPLs you need, and paste them into your own plug-ins. Then you can hack away at the individual PiPL field values using ResEdit's hex-editor option.

Very important: Remember that it's the 'name' field in the 'PiPL' resource (not the name of the module on disk!) that determines what name your plug-in has in the Filter menu when Photoshop is running. Also, it's the 'catg' field of the 'PiPL' that determines which submenu (Noise, Pixelate, Other, etc.) your plug-in shows up in. Be sure to set these fields up correctly if you want your plug-in to "show up" properly at runtime.

How Plug-Ins Work

When Photoshop invokes a plug-in, it calls the plug-in's entry point - which, on the Mac, must be named "main()" and must occur as the first function in the module - with a selector value indicating the type of action the plug-in is expected to perform. The prototype for main() looks like:

pascal void main (const short selector,
            FilterRecord *filterParamBlock,
            long *data,
            short *result);

The filterParamBlock will contain image data appropriate to the selector value as well as hooks to a variety of host callbacks.

The data pointer is for the plug-in's use; normally, you set this to a Handle to the plug-in's own global data. The host maintains the data value across calls to main(). Obviously, if you stuff a Handle here, you should lock it down at plug-in invocation and unlock it when the plug-in returns control to the host, so that the data can be relocated in memory as need be during host operation.

The result parameter is where the plug-in passes back a result code to indicate to the host whether the operation in question (i.e., the one requested via the selector value) encountered an error or returned normally (with noErr). See "Error Reporting," below.

The FilterRecord data structure is an enormous, 200-field structure containing references to a bewildering amount of state information about the current image (or selection), plus hooks to various callback services. An entire article (or small book!) could be devoted to explaining the various fields in this colossal data structure. Fortunately, you'll seldom need to use very many of the fields of this structure, although you should definitely spend some time studying it (see Table 7-2 in the API Guide in Adobe's SDK). Some key fields are shown in Figure 1. The main thing to know is that this structure is the primary vehicle by which the host and plug-in communicate with each other. Everything you might want to know about the current image is available to you through this structure, or else through one of the callbacks associated with this structure. We'll have occasion to refer to the FilterRecord structure frequently.

The following is a partial listing of fields of the FilterRecord data structure. (See the Adobe SDK docs, as well as PIFilter.h, for complete info).

Field Name	Explanation
abortProc	This field contains a pointer to the TestAbort callback function.
progressProc	Function pointer to progress bar callback. Call this during main processing loop. Takes two long-int arguments: current index, and total.
parameters	If your plug-in filter has any parameters that the user can set, you should allocate a relocatable block, store the parameters in the block, and store the block's handle in this field, after your setup dialog returns.
imageSize	This is a Point giving the dimensions of the image or selection.
imadeMode	The mode of the image (RGB, CMYK, etc.); see PIFilter.h for #defines.
filterRect	The overall area (Rect) of the image or bounding box of the selection.
maxSpace	The maximum number of bytes the plug-in can expect to access at once.
bufferSpace	Set this to the number of bytes of memory the plug-in is planning on allocating for internal buffers or tables. Do this prior to the first call for data, in order to minimize scratch disk activity and gain performance.
planes	The total number of planes in the image, including alpha channels.
inLoPlane,	Set these values to tell the host the first and last planes you inHiPlane want to process.
outLoPlane, outHiPlane	Set these to tell the host the first and last output planes you intend to write to. You can write to a subset of inLoPlane/inHiPlane, if necessary. Most times, however, you will set these equal to inLoPlane/inHiPlane.
inRect	Set this to the bounds of the portion of image you are requesting pixels for. On each call to AdvanceState(), the host will examine this field and place the appropriate data in inData (below).
outRect	Set this to appropriate bounds to ask for access to the output buffer. On each call to AdvanceState(), the host will examine this field and point to the appropriately sized buffer in outData.
inData	Pointer to the input buffer. (The buffer will be provided for you by the host. You do not need to set this value!)
outData	Pointer to host output buffer. (User does not set this value!)
inRowBytes	Number of bytes in each requested row of data (may include pad bytes).
outRowBytes	Number of bytes in each requested output row of the output buffer (may include pad bytes).
isFloating	Boolean, set TRUE if selection is floating.
hasMask	Boolean, set TRUE if selection is non-rectangular.
maskRect	If hasMask is TRUE and you want access to mask data, set this to request a portion of the selection mask.
maskData	If you've requested mask data, this pointer will point to the mask buffer, which will contain 8-bit data bytes, with 255 = not selected and zero = selected.
maskRowBytes	Number of bytes in each row of the mask data.
advanceState	Pointer to AdvanceState() callback function.
depth (int32)	Gives the bit depth of the channel info (1,8, or 16). Photoshop 5.0+ only.

Figure 1. The FilterRecord structure contains 200 fields. These are some of the more important ones.

The overall flow of events for the plug-in's lifespan is shown in Figure 2. There are five phases of operation, corresponding to the parameter setup phase (which is where the user gets to manipulate filter parameters via sliders, text boxes, radio buttons, or whatever), the "preparation" phase (where memory can be allocated or initializations done, as needed), the "Start" phase, the "Continue" phase, and a "Finish" phase (where cleanups can be done). The corresponding selector values are:

#define filterSelectorAbout        0
#define filterSelectorParameters   1
#define filterSelectorPrepare      2
#define filterSelectorStart        3
#define filterSelectorContinue     4
#define filterSelectorFinish       5

Writing a plug-in is mainly a process of writing handler functions to take care of each of the above selector cases. Obviously, if main() is called with a selector value of filterSelectorAbout, your plug-in should vector to its "About" handler so as to pop an About box, either directly or through the DoAbout() function defined in DialogUtilities.h.

Figure 2. Flow of control of an image filter plug-in.

When Photoshop calls your plug-in for the first time, it will generally be with a selector value of filterSelectorParameters. If your plug-in has any parameters that the user can set, this is the time to prompt the user (i.e., pop your setup dialog) and save the appropriate parameter values in a relocatable memory block whose handle is stored in the parameters field of the filterParamBlock. (Photoshop initializes the parameters field to NULL when starting up.) Now is also a good time to write the user's parameter settings to disk (or to a resource), if you want the plug-in's values to be persistent across Photoshop sessions, as most users expect. This is not the time to do initializations or allocate memory; these things should occur when main() is called with a selector value of filterSelectorPrepare.

When main() is called with filterSelectorPrepare, you should initialize any variables that need initializing, preflight or allocate memory as required, and return the appropriate result code (noErr if everything is proceeding okay, or else the appropriate standard MacOS system error code or one of the error codes given in PITypes.h or PIFilter.h). If you won't be allocating any large buffers in your "Start" routine (below), you should inform the host of that fact by zeroing out the bufferSpace field of the FilterRecord. If you will be allocating big buffers in your "Start" routine, set bufferSpace to the number of bytes you think you'll need so that Photoshop can try to have the memory ready for you when your "Start" routine gets called.

Assuming everything has gone okay so far, the next call to main() will be with a selector value of filterSelectorStart. At this point, the very first thing you should do is validate your parameters (and if need be, re-pop the parameter-setup dialog). When the user hits Command-F or chooses "Last Filter", the host does not call filterSelectorParameters, but instead proceeds directly from filterSelectorPrepare to filterSelectorStart; and as a result the user is not shown any param-setup dialog. Rather, the plug-in is handed whatever parameter values are stored in the parameters field of the filterParamBlock. If those values are garbage, now's the time to find out. Validate your parameters before proceeding.

Some plug-ins can do everything they need to do in one operation, in which case the primary action can be accomplished in the filterSelectorStart phase (and the dispatch function for filterSelectorContinue can be empty). This is generally true for plug-ins that make use of the AdvanceState() callback (see further below). The conventional way of doing things, however, is to put your "main processing loop" in the filterSelectorContinue dispatch function (see below).

The filterSelectorFinish phase is where you should deallocate memory and do any other cleanups that might be necessary at plug-in termination.

Error Reporting

Every call to main() expects that the result parameter will contain (after returning) an error code indicating whether the plug-in encountered an error or finished this phase of its job normally, which is to say with noErr. If a problem is encountered in any phase of operation, it's important to notify the host during that phase so that main() isn't repeatedly called with new selector codes. Note that if you return a negative result (corresponding to one of the standard System error codes, or else one of the errors defined in PITypes.h), Photoshop will report the error and its meaning to the user for you. If you want to report the error to the user yourself (for example, if the error has a meaning that's understood only by the plug-in), you can pop your own custom alert box and send a positive result code back to the host. A positive result signals Photoshop not to put up any additional alert boxes.

With Version 4.0 (and up) of Photoshop, you can handle error reporting in one other way, and that's to paste a value of errReportString into result and put an explanatory Pascal string in the errorString field of the FilterRecord. When you do this, Photoshop throws an alert saying "Cannot complete operation because <string>."

Note that filterSelectorFinish is not called if filterSelectorStart returns an error, but is guaranteed to be called if filterSelectorStart doesn't produce an error. This is to give your plug-in a chance to do any necessary cleanups after processing a portion of an image. Just remember that if an error occurs in filterSelectorStart, you should do any needed cleanups right then and there, because filterSelectorFinish won't be called.

Normally, you'll call TestAbortProc() - one of the host callbacks - repeatedly during your main processing loop to catch "Command-Period" cancellations by your user. (This also has the effect of causing the secondhand on the watch cursor to keep ticking.) Any time TestAbortProc() returns TRUE, you should stuff a result code of -128 and return. Photoshop will terminate the plug-in operation without further ado.

To recap:

1. Always report errors as soon as they happen. Do not "accumulate" error conditions so as to report them at the "Finish" phase.
2. If an error occurs in the filterSelectorStart phase, do your cleanups right away because filterSelectorFinish may not be called.
3. If you want Photoshop to tell the user about an error, return a negative number appropriate to the error. (Use one of the MacOS system error codes, or else the appropriate error code defined in Adobe's include files.) If you want to report the error yourself, return a positive error code. In general, you should let Photoshop report as many error conditions as possible, because this way you can let the host deal with localization and Unicode conversion issues. (If your plug-in is running on a Japanese user's machine, errors will be reported in Japenese, etc.)
4. The most common error condition is a user abort. Return -128 for this condition.

Processing the Image

As a general rule, you'll make your first request for pixels in your "Start" routine, rather than the "Continue" routine. That's because Photoshop relies on the values placed in the inRect, ourRect, and maskRect fields of the FilterRecord to know when you're done processing. If you leave these fields set to NULL, your "Continue" routine will never get called! So when main() is called with a selector value of filterSelectorStart, the thing to do is set inRect and outRect to some subset of filterRect, representing the initial chunk of image data you wish to work on. (The filterRect field of the FilterRecord contains the overall dimensions, in pixels, of the image or selection. If the user has chosen a selection, the filterRect represents the bounding box of the selection. That means that if the user has lassoed a non-rectilinear piece of the image, the furthest horizontal and vertical extents of the selection are given in the filterRect. And by the way, in case you're wondering, the answer is yes, Photoshop does mask your filter's output properly for you, automatically, so that your filter changes only those pixels within the lasso area.)

If you set inRect and outRect to some subset of filterRect, the inData and outData fields of the FilterRecord will contain the requested portion of the image (or of the selection, as the case may be) when main() gets called with filterSelectorContinue. Check these data fields, and if they are non-NULL, proceed with image processing. At the end of your "Continue" function, if there are any unprocessed portions of the image remaining, you should reset inRect and outRect to a new subset of filterRect before returning. Photoshop will then examine inRect and outRect, and if they are non-NULL, will call main() with the "Continue" selector again. This cycle will repeat over and over again until an error occurs or you finish processing the image, whichever happens first.

Chunking the Image

Most of the time, you'll set inRect and outRect to some subset of filterRect. The question is how much of the image to request at one time, and what form (e.g., raster lines versus tiles) to request it in. The former depends, to some extent, on how much free RAM is available. You can get a rough idea of this by checking the maxSpace field of the FilterRecord. If space permits, you can request the entire image at once. (Even if space doesn't permit, you can still request it, and Photoshop will generally still give it to you, spooling out to the Scratch Disk as necessary.) But as a rule, requesting the whole image at once is not a good default behavior, because this will result in unnecessary disk thrashing and/or memory warnings. Also, keep in mind that you want to call TestAbortProc() frequently enough so that the user can cancel out of an operation in midstream. By working on the image in small chunks, you can do one TestAbortProc() call per chunk, and keep your main processing loop clean, yet also give the user ample opportunity to abort. (After an abort, Photoshop restores the original image or selection for you, by the way. There's no need for special action on your part.)

The question of whether to chunk the image by square tiles or by vertical or horizontal strips is partly an aesthetic choice, but also depends on the nature of the operating system (multiprocessor systems tend to choke on strips but do quite well with tiles). Probably the most important consideration is simply the nature of the processing algorithm. If you're trying to do a vertical motion blur (averaging adjacent vertical pixels), you won't want to request horizontal raster lines. Likewise, if you're attempting a Fast Fourier Transform of whole columns or rows, you won't derive any benefit from 8x8 tiles. But conversely, certain "area operations" (like the Discrete Cosine Transform done in JPEG compression) demand that you chunk the image in tiles. Let common sense be your guide here.

Regardless of how you chunk up the image, be sure to call TestAbortProc() and ProgressProc() frequently - but not so frequently as to slow down an otherwise tight loop. The ProgressProc() callback takes two long integers as arguments: one number representing the current index of the main operation, and another representing the total number of iterations. By calling this function, you let Photoshop display a progress bar and watch cursor while your plug-in cranks along. (Photoshop automatically suppresses this bar for short operations.) If your user hits the Cancel button in the progress-bar window, TestAbortProc() will return TRUE and you can exit with an error code of -128.

Once you've processed the entire image (or selection), be sure to set inRect, outRect, and maskRect to NULL so that the host knows to proceed to the "Finish" phase.

The AdvanceState Callback

With Version 3.0 of Photoshop, the API provided a new callback function called AdvanceState(), as in "advance the state of the operation." What this function is designed to do is let the plug-in drive the host through the buffer-reloading (or "image-chunk bucket brigade") process, so that program control doesn't have to flip-flop back and forth between the plug-in and the host. This serves two purposes. First, it greatly facilitates the use of "proxies," or preview windows in the setup dialog. (Without some means of driving the host through the buffer-reload operations, the plug-in would have to somehow pre-buffer the image before entering the setup dialog, in order to show a preview.) Secondly, it eliminates a certain amount of overhead due to unnecessary flip-flopping of control between plug-in and host. Without AdvanceState(), a plug-in is called from - and must return to - the host for each chunk of image data. With AdvanceState(), a plug-in can complete its image processing within a single host call - and with any luck, the main pixel-tweaking loop can just stay in the host machine's instruction cache and rip.

Plug-ins that use AdvanceState() typically do all their processing in the "Start" phase and do nothing in the "continued" phase. There is no real need for a "continued" phase when AdvanceState() is used. In a sense, AdvanceState() should probably be called FetchMoreData(), or something similar.

If you decide to use AdvanceState(), always check its return value on each call. A normal return value is noErr. If you get something other than noErr, pass the return value back to the host in result, from main(). Do not call AdvanceState() again after a non-noErr return, because once an error is returned, the inData and outData pointers in the FilterRecord will be NULL, and if your plug-in tries to use them you'll be in Big Trouble. The most common non-zero return value for AdvanceState() is -128, indicating that the user cancelled out of the plug-in.

The precise behavior of AdvanceState() varies somewhat depending on the type of plug-in you are writing. Refer to Adobe's SDK doc files for details. Typical code for processing an image using AdvanceState() might look like Listing 1. This routine would be called from the plug-in's "Start" handler, after popping the user-setup dialog (if need be).

Listing 1: FilterTheImage()

FilterTheImage()

This shows how to drive the host through the pixel-acquisition loop using AdvanceState(). Typically, this function would be called from the plug-in's "Start" handler. The "Continue" handler would be empty.

int32 FilterTheImage( FilterRecord *fr )
{
   int32         err = 0;
   int16          i;

   // Tell host we want all planes...
   fr->inLoPlane = fr->outLoPlane = 0;
   fr->inHiPlane = fr->outHiPlane = fr->planes - 1;
      
   // We'll ask for raster lines...
   fr->inRect.left = fr->outRect.left =
         fr->maskRect.left = fr->filterRect.left;
   fr->inRect.right = fr->outRect.right = 
         fr->maskRect.right = fr->filterRect.right;

   // Do not subsample image
   fr->inputRate = long2fixed (1L);
   fr->maskRate = long2fixed (1L);

   // Loop over all raster lines...
   for (i = fr->filterRect.top; i < fr->filterRect.bottom; i++)
      {
      
      // Update our request...
      fr->inRect.top = fr->outRect.top = fr->maskRect.top = i;
      fr->inRect.bottom = fr->outRect.bottom = 
               fr->maskRect.bottom = i + 1;
      
      // Request the image data chunk...
      err = AdvanceState ();
      
      if (err != noErr || (*fr->abortProc)() == TRUE)
         goto done;

      // Let host show a progress bar...
      (*fr->progressProc)(i - fr->filterRect.top, 
               fr->filterRect.bottom - fr->filterRect.top);
      
      // Process the pixels...
      err = ProcessFilterRect (fr);
      
      if (err != noErr)
         goto done;
      }
      
   done:
   SetRect (&fr->inRect, 0, 0, 0, 0);
   SetRect (&fr->outRect, 0, 0, 0, 0);
   SetRect (&fr->maskRect, 0, 0, 0, 0);

   return err;
}

Notice that we test for a user abort and call the progress-bar callback once every trip through the loop (i.e., once per raster line), which ought to be more than enough. Also, note that the block of code under "done" tells the host that no more pixels will be requested; hence we can move on to the next phase of operation (in this case, the empty "Continue" phase). The actual pixel manipulations are done in ProcessFilterRect(). Bear in mind, we may be operating on the whole image or we may just be operating on a selection. In either case, filterRect contains the bounds.

The initial lines of code talking about planes (inLoPlane, outLoPlane, etc.) require a bit of explanation. The inLoPlane and outLoPlane fields of the FilterRecord are what the plug-in uses to communicate to the host which image channels (or planes) we want to work on. The host has to be told this prior to fetching any image data so that it knows how big the input and output buffers need to be. Basically, the lines of code shown in the example request all available channels for the active layer. To really understand what's going on here requires a bit of a digression on how Photoshop organizes image data. So let's stop and talk about that.

How Image Data Is Organized

Image-filter plug-ins can operate on any portion of an image (individual colors, alpha channels, transparency characteristics, etc.), or all portions, either one plane at a time or - more commonly - all in one loop, with plane data interleaved. To request this data, you set the inLoPlane and outLoPlane fields of the FilterRecord to appropriate values prior to the first call for pixel data. To know how to do that, we have know how Photoshop organizes image data.

A Photoshop image is actually a meta-image or "composition" made up of component images called layers. Each layer is like a fully self-contained image with its own pixel data. The number of layers in an image is arbitrary (up to a maximum of 100, in Photoshop 5.0), but the exact number needn't concern us because a filter can never operate on more than one layer at one time. That's because the user can never operate on more than one layer at once. In Photoshop, only one layer can be active - which is to say, editable - at any given moment. (They can all be visible at once, but the user can only work on one layer at a time.) The filter therefore has access to only the active layer (or a selected portion of it).

Note that although an image might contain many layers, each layer has to be of the same resolution and image mode. There's no such thing as an image with one RGB layer and one CMYK layer, for example.

If you think of each layer as an image unto itself, you can understand the next step of the hierarchy: channels. Each layer is made up of channels containing the color information for the layer. In the old days (before version 3.0), Photoshop images generally had either 1, 3, or 4 channels, depending on whether the image was a bitmap, RGB, or CMYK format. Things are more complicated today, because now users expect to be able to add alpha channels and spot colors to the underlying "composite" channels. At this writing, Photoshop permits up to 24 channels per image.

Just as all layers of an image have to be of the same image mode (RGB, CMYK, grayscale, Lab color, etc.), they also all have to have the same number of channels. This makes sense, since otherwise, if you could have more channels in one layer than in another, you could (in theory) have an RGB layer and a CMYK layer in the same image.

Since layers can mask each other and have opacity characteristics, and since it matters a great deal how the various layers are ordered, all of this information has to be preserved in a consistent manner. Photoshop's default behavior is to devote one 8-bit byte to each channel and each transparency or mask attribute. Collectively, the channel, mask, and transparency attributes are called planes of data, and if you request all the planes at once, Photoshop gives the bytes to you in one big array, with plane data interleaved from bottom to top in the following order:

1. Composite color bytes (for example, red, green, and blue bytes for an RGB image).
2. Opacity pre-multiplier byte (255 being fully opaque).
3. Mask of "normal" polarity (255 being fully masked).
4. Mask of "inverted" polarity (255 being fully transparent).
5. Alpha channels.

This is the default order in which channels are created. With Photoshop 5.0, users can reorder channels by dragging them up or down in the channel list. When that happens, the bytes change storage orders accordingly. You can enforce the above ordering, however, at plug-in runtime by setting the wantsAbsolute field of the FilterRecord to TRUE before requesting data. This is a good habit to get into.

So if you're dealing with a CMYK image that has two alpha channels and a normal-polarity layer mask operating on the target layer, you would expect seven bytes of image data per pixel - cyan, magenta, yellow, black, mask, and two alpha channel bytes. See Figure 3.

Figure 3. How Photoshop organizes channel data.

The number of planes (or data bytes) per pixel can always be checked by inspecting the planes field of the FilterRecord.Thus, if you want to request all data bytes, you would set inLoPlane to zero and inHiPlane to plane - 1 before making your first call for pixel data. Likewise, if you know you are dealing with an RGB image and you just want to retrieve the "color" bytes, you can set inLoPlane to zero and inHiPlane to 2 (so that you fetch color planes zero, one, and two - R, G, and B).

Since Photoshop currently recognizes at least 14 differrent image modes (with from zero to eight data bytes each), you should plan on having your plug-in consult a lookup table at runtime to determine how many "color" bytes are available in an image. The imageMode field of the FilterRecord can be inspected to determine what kind of image you're dealing with.

Because channel data is interleaved, pixel processing code usually looks something like this:

for (i = 0; i < rows; i++)
{

      srcPtr = (unsigned8 *) fr->inData + (i * fr->inRowBytes);
      dstPtr = (unsigned8 *) fr->outData + (i * fr->outRowBytes);
          
      for (j=0; j < columns; j++)
      {      
         for (plane = 0; 
            plane < bytesInMode[ fr->imageMode ]; // lookup table
            plane++)
   
            dstPtr [plane] = FiddleWithPixel( srcPtr[plane] );

         // skip over unprocessed planes...
         srcPtr += fr->inHiPlane - fr->inLoPlane + 1;
         dstPtr += fr->outHiPlane - fr->outLoPlane + 1;
      }
}

Bytes	Mode (see #defines in PIGeneral.h)
1	plugInModeBitmap
1	plugInModeGrayScale
3	plugInModeIndexedColor
3	plugInModeRGBColor
4	plugInModeCMYKColor
3	plugInModeHSLColor
3	plugInModeHSBColor
0	plugInModeMultichannel
1	plugInModeDuotone
3	plugInModeLabColor
1	plugInModeGray16
3	plugInModeRGB48
6	plugInModeLab48
8	plugInModeCMYK64

Figure 4. Number of bytes of color data per pixel, for various image modes.

Memory Allocation

If you intend to allocate a lot of buffers (and especially if you want your code to be portable across platforms), you should use the Buffer suite and/or Handle suite callback services (documented in Adobe's API Guide) rather than MacOS calls. This is important for two reasons. First, not every operating system has an equivalent to MacOS "handles"; the Handle suite implements an equivalent platform-independent facility. Secondly, if you use host callbacks, you'll get all the benefits of the host's virtual memory management system (instead of blindly eating away at the application's heap), and Photoshop will use scratch disk space far more efficiently. If you insist on doing raw calls to NewPtr() or NewHandle(), you may force the host into a lot of unnecessary double-swapping operations. Also, there's the fact that the MacOS doesn't handle memory management (where large buffers are concerned, especially) as efficiently as Photoshop does. (Photoshop uses its own internal memory management routines, bypassing the operating system.) For best overall performance, use the callback suites.

Debugging Tips

Debugging image filters isn't difficult, because problems in the pixel-processing portions of your code will more often than not be evident in the onscreen appearance of the image, whereas problems of a structural nature with the plug-in itself can usually be quickly isolated to one of the handler functions, and from there to a subroutine containing (hopefully) relatively few lines of code.

Some tips to keep in mind as you work:

• Be sure your plug-in shows up in the proper submenu. If it doesn't load at all, it may be because of a problem with the 'vers' field in the 'PiPL' resource. If it doesn't show up with the proper name or in the proper submenu, it's because of problems with 'name' or 'catg' fields in the 'PiPL' resource.
• Always test your plug-in against as many image mode cases as possible (RGB, Lab color, grayscale, etc.), with and without lasso selections, with/without alpha channels, etc.
• Don't assume that all color images are RGB! Don't just stuff pixel values with red, green, and blue values, because if the image is CMYK or Lab color, you won't get the expected results. If you need to convert colors into and out of different spaces, use the utilities in the Color Services callback suite. (See the SDK docfiles.)
• If you don't want your plug-in to handle certain image mode cases, such as Lab color, configure the 'PiPL' resource appropriately so that Photoshop enables and disables your plug-in under the proper conditions.
• Because of all the array-based processing that goes on in a plug-in, one of your biggest potential bug sources will be array out-of-bounds conditions and/or null pointer access.

Since the host handles file I/O, file conversions, event-loop overhead, error reporting, and progress bars, your code should be relatively streamlined and easy to troubleshoot. But for heavy-duty bug removal, you'll want to be able to step through the code with a debugger at runtime. Happily, you can do that with the Metrowerks Debugger, if you follow these steps:

1. Compile your project, then drag the .SYM file out of the development directory to the Desktop.
2. Double-click the .SYM file to run the Metrowerks Debugger.
3. When it asks "Where is the executable?" select your plug-in in the Photoshop plug-in folder.
4. Drag the Photoshop desktop icon on top of the Metrowerks Debugger icon to link Photoshop to the debugger.
5. Launch Photoshop.
6. Set your breakpoints in your .SYM window in the debugger.
7. Bring Photoshop to the front and run your plug-in. When your first breakpoint is hit, the plug-in will drop into the debugger.

Note that you do not have to relaunch Photoshop in order to load and test new "builds" of your plug-in. Photoshop reloads plug-ins dynamically as they are needed. But Photoshop will not rescan the hard disk for brand new plug-ins during a work session; for that, you have to quit and relaunch.

Conclusion

We haven't managed to cover everything just yet, of course. For example, we have yet to talk about how and when to pop a movable modal setup dialog, or how to implement a realtime proxy (image preview) inside a dialog. We also haven't discussed the various callback suites in any detail, and there's a bug or two in Adobe's SDK example code that could bite you if you're not wary. (Hint: One involves a buffer out-of-bounds condition.) We'll talk about these matters - and a lot more - next time, when we do an in-depth code walkthrough of an honest-to-gosh image filter.

In the meantime, if you're an image- or signal-processing junkie and you want to test new ideas quickly, look into writing Photoshop plug-ins. You get all the benefits of a stable host program (one that does memory management and buffering chores for you, automatically) without having to write, debug, or maintain user-interface code (except, possibly, for a single dialog). You don't have to worry about file format conversions, monitor modes, GWorlds, buffer allocations, device dependencies, Apple Events, or dozens of other concerns that weigh graphics programmers down. Instead, you can concentrate on pixel-poking, which is what graphics programmers do best.

---

Kas Thomas <tbo@earthlink.net> has been a Macintosh user since 1984 and has been programming in C on the Mac since 1989. He is the author of a number of Photoshop® plug-ins, including the four-star-rated (by ZDNet) Callisto 3D shareware plug-in, available at http://users.aol.com/Callisto3D and elsewhere.