Volume Number: 15 (1999)
Issue Number: 3
Column Tag: PCI For Mac Programmers

PCI for Mac Programmers

by Larry Barras

Understanding the PCI Expansion Bus

In 1995, Apple introduced the PCI (Peripheral Component Interconnect) expansion bus to the Power Macintosh line, replacing the older NuBus slots. PCI is now the dominant bus standard for add-in cards; such as disk controllers, video cards, network interface cards for desktop PC's and Power Macs. By adopting PCI as the standard for expansion slots, Apple embraced the wider variety and lower cost hardware available on the PC. Almost all PCI hardware functions on the Mac, provided there is software to drive the device.

This article is designed to give a programmer's overview of the PCI bus, as implemented in the Macintosh computer. PCI is a cross-platform standard and some concepts of the PCI interface will be unfamiliar to Mac programmers. Developers who have experience on PC's will be unfamiliar with some of the Mac's unique features as well. Once the concepts and differences are cleared up, the actual programming is fairly simple. Native Mac drivers usually require no assembly code, and many useful routines are provided in the Mac OS. Hopefully, this article will encourage more developers to produce Mac drivers for PCI devices.

Take any PCI card and plug it in your Mac, and power-up. If the card wasn't specifically made for the Macintosh, chances are not much happened. But this is not necessarily a bad thing. Your Mac did boot after all. The card requires a software driver to function on the Mac. There are two types of drivers for PCI Cards. Boot driver is loaded from the PCI Cards firmware and executed when the Mac hardware in initialized. A Mac specific device driver is loaded from the hard drive when the Mac OS is initialized. Only cards needed when loading an operating system require firmware based drivers. Usually, this means video cards and disk controllers for a bootable device.

Firmware Driver

The common MS-DOS PC boots with native x86 binary code. Writing PC-Compatible boot code for PCI cards is a difficult art. Many aspects of the low-level PC were never well documented, and many variations exist today. Most expansion BIOS ROMS devote a good portion of code to figuring out what kind of hardware it is running on, and how to deal with it. Obviously, a PowerPC processor cannot directly interpret x86 code. If a card requires boot time support, like disk controllers and video cards, the card will need a ROM with boot code in it.

The Mac had the opportunity to start with a clean slate. The boot environment is well defined, and boot-time code need not worry about machine-specific conditions. While most PCs look the same at boot time, Macs may have tremendous differences in things like video adapter addresses, disk controllers, network adapters and so on. The raw hardware looks different, making it difficult to program in the traditional, machine-specific manner.

Open Firmware

In order to ease the card developer's burden, allow for future changes in hardware and provide a positive end-user experience, PCI Macs utilize Open Firmware at boot-time. Open Firmware is based on the Forth language and provides a processor-independent way to implement firmware on expansion cards. Instead of native binary PowerPC or binary x86 code, Open Firmware uses processor independent Forth code. An expansion ROM with Open Firmware code on a PCI card can supply information about the device to the host computer and can even provide basic drivers for bootstrap devices, until an operating system can load and provide OS-specific drivers for the device.

Open Firmware originated at Sun Microsystems. Sun needed a way to build computers and expansion cards that worked on all the different processors they were working with. Until Open Firmware and the Open Boot specification, computers booted via firmware written for a specific processor family. Sun was building machines with 68K, Sparc and x86 processors. Previously, every card needed firmware coded for each processor family and system design. Forth was a natural choice due to its compactness and availability on nearly every known processor.

Open Firmware also has an interactive Forth console. The console is important for developers because it provides a low-level interface before the Mac ROM is loaded. Some Macintosh computers use the keyboard and display; other models may require a terminal on a serial port to access the console. Using the console gives the developer a chance to intercept the boot process and debug code before an operating system or something like MacsBug can load.

When a PCI Mac is booted, Open Firmware loads from the Mac's ROM and initializes the main board. Open Firmware then scans the PCI bus for any user-installed cards and scans their configuration registers. The configuration registers indicate what resources, such as interrupts and memory addresses, the card needs, and whether there is a boot ROM present. When Open Firmware code is executed, unrecognized ROM code is simply ignored.

During the scanning process OpenFirmware builds a table of devices and resources in the system called the Device Tree. The information in the Device Tree is carried over to the Mac OS in the Name Registry. Mac OS drivers and applications use the Name Registry to find things like PCI cards, where they are located in memory, what resources they require, where their interrupts are serviced, power requirements and so on.

Apple defines three levels of firmware driver support; no firmware, minimal firmware, and full firmware. A card with no firmware support on-board leaves generic information in the name registry, and requires a disk-based driver. Limited firmware includes some Open Firmware code to install specific properties in the name registry, allowing your disk-based driver to positively identify your hardware. A Mac OS runtime driver might also be included in partial support scenario, but such cards will be limited to booting only the Mac OS. Full firmware support includes a Mac OS driver and an Open Firmware driver in Forth to allow loading of an alternative operating system like Mac OS X Server. An Open Firmware driver makes the device available for the entire booting process, even before an operating system is loaded. Again, anything that needs to participate in the boot process needs full Open Firmware support.

Apple recommends developers implement minimal firmware support, and supply name and resource information, but it is by no means required. Ordinary PCI cards, like data acquisition cards, frame grabbers, and even the popular 3dfx Voodoo game cards don't have any ROM at all.

The Name Registry

All the information that Open Firmware records in the Device Tree is passed along to the Mac OS in the Name Registry. The Name Registry is a simple tree-like database, which records system information. The Name Registry is unique to the Macintosh and greatly eases driver development.

On most other platforms, a PCI driver developer needs to know lots of detail about how the hardware works. Often one must resort to difficult techniques in order to configure the hardware, reserve memory, determine the CPU speed and so on. Under the Mac OS, things are a lot easier. Using the Name Registry, your device driver can easily determine whether your hardware is present and what resources are assigned to it, and other information like the PCI clock frequency.

The Name Registry provides functions to locate specific instances in the device tree, retrieve properties such as address space and interrupt-tree location that are allocated to your hardware. Your device driver will use the Name Registry to retrieve the physical and logical addresses, interrupt resources and other information about your device. The utility "Display Name Registry," part of Apple's PCI Driver SDK, is useful for looking at the Name Registry.

The first step is to grasp what PCI offers on any computer platform. PCI devices present well-defined means for identification, configuration and operation. Identification and configuration are handled via a unique addressing mode. Since the PCI standard is cross-platform, it offers two means for transfer of data, IO-mode addressing, common for Intel x86 processors, and memory addressing, usually found on other processor families like the PowerPC. Either IO or memory-mapped addressing may be provided, sometimes both. However, due to the popularity of the Intel platform, many devices only offer IO-mode addressing. Unlike the Intel family processors, the PowerPC doesn't offer special IO addressing. But, as I will explain later, some clever hardware built into the PowerMac takes care of this problem.

Under PCI, there are three relevant address spaces: Configuration Space, Memory Space and IO Space. Configuration Space is how the card is identified and controlled. Memory Space acts like any other physical memory address space. IO Space is a special kind of address space, devoted to input/output on some processor types like the Intel x86. To access these address spaces, the Mac provides helpful managers. The function calls are documented in "Designing PCI Cards and Drivers" from Apple Computer.

Configuration Space

Configuration Space contains the registers that identify and control the card itself. This space contains a total of 256 bytes for each card. The PCI standard defines part of these addresses for control and identification registers. Registers above offset 0x3F are device-specific and may be used by the card to control hardware on the card.

Configuration registers are not mandatory. Vendors are free to implement optional registers as they see fit. Registers above 3Fh are device-specific. Refer to the vendor's documentation to determine device-specific register functions and whether any are implemented.

The control register lets the host turn the card on and off, by enabling and disabling the card from decoding memory or IO addresses. Cards are left in a disabled state by default.

The base address registers point to a base address in memory or IO space. The hardware on the card will respond to this base address when enabled. The lower three bits in the base address registers are flags that tell the host whether the address is in IO or memory space, and how much room to reserve for the hardware. The flags are tested by writing all ones and all zeroes into the register and examining the result. Address space is assigned by writing a new value into the base address registers. On the Mac, all of this is done for you by the operating system. There is no need to reassign addresses or alter the contents of the base address registers.

The ROM address register is similar to the base address registers. The value in the ROM address register decides where in physical address space the ROM resides. The least significant bit is a switch. Setting bit-zero high enables access to the ROM. Both the ROM-enable bit of the ROM base register and the memory addressing enable-bit of the command register must be set before attempting to access the ROM. The Mac's Expansion Bus Manager calls are the only way to access configuration registers on the Macintosh.

Memory Space

Memory space is familiar to most Mac programmers. Traditionally I/O devices on a Macintosh are accessed through fixed memory addresses. This is called "memory-mapped IO." Serial UART chips, video controllers, SCSI controller chips, and other devices are controlled via regular memory addresses. PCI allows for decoding of memory address space. This kind of addressing is most common with devices like frame-grabbers and video cards, which often have frame storage memory onboard.

The card decodes memory by the physical address on the system bus. Because of virtual memory, this is not the same as the logical address. The logical address used by software is remapped to a physical address on the system's physical address bus by the memory controller. What appears as one contiguous block of logical memory is really scattered throughout physical memory locations. The Mac's Name Registry provides properties defining both the physical and logical addresses assigned to a PCI device. Also, the Driver Services Library provides functions to convert the physical address to a logical address and vice versa.

IO Space

IO Space may feel unfamiliar to Mac OS programmers. Some processor families, notably the Intel x86, use special IO modes and address to talk to hardware devices. Intel x86 and x86-clone computers use IO addressing to communicate with practically all hardware other than memory. The x86 IO scheme uses a unique addressing mode to send or receive data to or from a device. The x86 can address up to 65535 devices using a separate 16-bit logical-address scheme, aside from the memory address-space. This method for handling hardware on a PC under DOS or Windows arose partly from the heritage of the x86, and partly because of the segmented memory unique to the x86. If you've ever installed hardware in an Intel-based PC, you've probably run into the IO address or "ports" as a range of hex numbers that had to be reserved for some device. Most PCI devices are intended for the Intel architecture and consequently use IO address space.

The PowerPC processors do not provide device-only address modes. Since PCI supports IO cycles, and many PCI devices also support IO cycles, this poses a problem. Fortunately, the PCI hardware in the Mac synthesizes device-IO address cycles. A 64K block of memory address space is set aside for this purpose. The logical addresses are assigned automatically by the Mac's boot-up code, and stored as properties under the card's node in the name registry.

The Expansion Bus Manager provides function calls to directly generate IO-device cycles to the PCI bus. However, these functions carry significant overhead. In every case it is better and faster to get the logical address properties from the Name Registry, and let the PCI controller hardware do all the work. The unique IO cycle hardware makes it possible to write device drivers completely in C, without resorting to assembly language for esoteric addressing modes.

The other really strange thing about IO space is that a single address may be 1, 2 or 4 bytes wide. Reading a 32-bit value at address 0xFF00 is not the same as reading four bytes at 0xFF00, 0xFF01, 0xFF02 and 0xFF03 sequentially. Don't worry about it, because the hardware handles it all invisibly. If the device in question is 8, 16 or 32-bits wide, just read or write to it in that length, even if the very next device register is only one address away. It just works.

Endian Issues

One problem Mac programmers run into when dealing with PCI is something called "little-endian." A little-endian system defines multi-byte values so the address points to the least significant byte, and the last address points to the most significant byte. Macintosh computers are all "big-endian," where the address of a multi-byte value is the most significant byte, and the least significant byte is the last. An easy way to remember which endian is which is the sentence, "endian-little hate We." The Expansion Bus Manager provide byte-swapping functions, or you can define macros in C which are faster. PowerPC and Motorola 68k are both big-endian processors, where as the x86 class processors are little-endian. Neither is superior to the other, just different. Big-endian is more intuitive and easier to understand. Little-endian is more difficult to envision, and you have to mentally byte-reverse it while viewing it in a debugger. Note that the bit ordering does not change. A byte looks the same regardless. Only the ordering of bytes changes in multi-byte values.

The most important thing to remember is not to confuse the two and write a big-endian value into something that expects little-endian, and not to read something in little-endian and evaluate it as big-endian.

Endian example:

char oneByte = 0x0A;
short twoBytes = 0x000B;
long fourBytes = 0x0000000C;

Looking at these variables in memory would appear like this in big-endian:

variable address   byte-value
----------------   ----------
oneByte  +0          0x0A

twoBytes +0          0x00
     +1              0x0B

fourBytes +0         0x00
     +1              0x00
     +2              0x00
     +3              0x0C

In little-endian format, the bytes are arranged like this:

variable address   byte-value
----------------   ----------
oneByte  +0          0x0A

twoBytes +0          0x0B
     +1              0x00

fourBytes +0         0x0C
     +1              0x00
     +2              0x00
     +3              0x00

Most PCI device documentation specifies three different argument sizes; byte, word and double word. These correspond to 8-bits, 16-bits and 32-bit values. You will run into these terms frequently. Remember that the values specified are little-endian in most cases.

Now that the concepts have been covered, I will present a simple practical example. The example program will search for a specific PCI card by its vendor ID, check to see if the card has any ROM on board, and dump the first hundred bytes in the ROM to the console. This program will work on most off-the-shelf generic PCI cards. This example probably will not work on a card developed for the Mac, since these usually have firmware to set up the name registry with information private to the card's driver, instead of generic information. But if you borrow a generic card with no ROM, or one intended for an Intel PC, this code will work.

The first step is to search for the card by its unique Vendor ID. We can get the vendor ID from the manufacturer, or use a tool like "Display Name Registry" from Apple's PCI driver SDK. The search is conducted by iterating through the name registry until we find a match.

// On entry, the parameters are propertyName = "name" and
// propertyValue = "pci1000,123" where 1000 and 123 are the
// device ID codes specific to your device.
// propertySize is the length of the C string "pci1000,123". 
// The function returns the found node in foundEntry parameter
// and returns an error code if an error occurs.

FindPropertyWithValue(const RegPropertyName *propertyName, 
                        const void *propertyValue, 
                        const RegPropertyValueSize propertySize, 
                        RegEntryID *foundEntry)
{
   RegEntryIter          cookie; 
   RegEntryID             theEntry; 
   RegEntryIterationOp    iterOp; 
   Boolean             done; 
   OSStatus             err = noErr;

   // Registry searches all work by creating search cookie, iterating
   // the name tree, then disposing the tokens.

   // first step is to initialize the entry to a known state. The
   // data types are opaque, so we must do this step.
   ::RegistryEntryIDInit(&theEntry);

   // create a registry search token or cookie.
   err = ::RegistryEntryIterateCreate(&cookie); 
   if (err != noErr) 
      return err;

   iterOp = kRegIterContinue; 

   // search for the desired property by name 
   err = ::RegistryEntrySearch(&cookie, iterOp, &theEntry, &done, 
                     propertyName, propertyValue, propertySize);

   if (!done && (err == noErr)) 
   { 
      *foundEntry = theEntry; 
   }
   else if (done) 
      err = done; 

   // after completing a registry search dispose of anything we created.
   ::RegistryEntryIDDispose(&theEntry); 
   ::RegistryEntryIterateDispose(&cookie);
    
   return err; 
}

Once we find the card by matching the name property, the function returns a NodeID that we use to refer to that instance of the device. If the rombase register shows a ROM installed, search the properties for this node to find the logical address to read it.

   // OK, we found the card, now lets find out if there's a ROM on it.
   
   LogicalAddress deviceAddress = 0L;
   ByteCount deviceByteCount = 0;   
   
   // The ROMBase address pointer is in PCI configuration space, at
   // offset 0x30.
   err = CPCIUtils::GetDeviceLogicalAddress( &gMyDevice, 
                                    0x30, 
                                    &deviceAddress, 
                                    &deviceByteCount);

Next, enable the card for memory operation, and enable ROM access on the card. Use the NodeID, and flip a bit in the configuration register, and the lower bit of the rombase register.

   // now enable the card for operation by setting bit one of the
   // command register.  The Command register is at offset +4 in
   // configuration space.  Bit one enables memory addressing of the
   // card. Bit zero enables IO cycle operation and Bit two enables
   // Bus Mastering (DMA) on the card.
    err = ::ExpMgrConfigWriteByte(&gMyDevice,
                                          (LogicalAddress) 4, 0x02);
   
    // now enable the rom for operation
    // once the card has memory access turned on, we must specifically
    // turn on the ROM address decoder on by setting bit zero of
    // ROMBase address register.
    err = ::ExpMgrConfigWriteByte(&gMyDevice,
                                          (LogicalAddress) 0x30, 0x01);

Last, just read out 100 bytes of memory, starting from the base address.

   // Well, there is a ROM, and we will print out the first 100 bytes
   // in the ROM.   
   if( deviceByteCount > 0 )
   {
   
      cout << "We have a ROM, and we will print out the first 100
      bytes now." << endl;
      cout << "Note the ROM signature in the first two bytes
         of hex 55 AA, as required in the PCI specification."
            << endl<< endl;
      cout << "Offset  Byte Value" << endl << endl;
   
      for( UInt32 i = 0; i < 100; i++)
      {
      
         UInt8 byte = * ( (UInt8*)deviceAddress + i);
         
         printf( "%d \t\t %02X \n", i, byte);
      
      }
   
   }

Most PCI cards will work in the Macintosh, if you write a driver to support it. The Mac's unique hardware and well designed driver support software make the task of writing drivers far simpler than writing drivers on other platforms. Because many interesting PCI cards are available for the Windows/Intel PC, Mac developers should see this as an opportunity to exploit low-cost PCI hardware by writing drivers and support code so these devices work on the Mac.

• Apple Computer. Designing PCI Cards and Drivers for Power Macintosh Computers. Addison-Wesley, 1995.
• PCI-SIG. PCI Spec. Rev. 2.1S. PCI Special Interest Group, 1997.
• Brodie, Leo. Starting Forth. Prentice-Hall, 1987.
• Shanley-Anderson. PCI System Architecture. Addison-Wesley, 1995.
• Apple Computer. Technotes: 1135, 1104, 1061, 1062, 1044, 1008

Larry Barras is a Senior Software Engineer with Perceptive Scientific Instruments in League City, Texas. Larry develops advanced chromosome imaging and analysis software for PSI's Power Gene line of Mac-based instruments. When he is not programming, you can find Larry and his Gibson hollow-body guitar at the local blues hangouts. You may email Larry at larrysb@aol.com.