Threaded News

By Jörg Langowski, MacTutor Editorial Board

MacForth Plus 4.0; NEON news

And here we are back with a “real” Forth column again. Just in time, it seems; only when I read on GEnie recently “It is too bad that he was seduced by the dark side of the force” about my recent excursions into other domains I realized that the true believers might have been disappointed. But please realize, there is only so much space in our magazine, and we have to try and cover a great variety of subjects.

A new version of MacForth Plus (v 4.0) has arrived here; with lots of improvements, notably faster execution and compilation, the possibility to generate stand-alone code, and some rudimentary but very efficient object-oriented extensions. Before I tell you more about it, I’d like to show you that there are still some people out there using NEON, and improving it. I received the following note from a reader in Australia:

“Dear Jörg,

I was delighted to read in your October column in MacTutor, that some people are still using NEON. I thought I was the only one! I enclose a copy of a letter I have sent to Bob Loewenstein, for your interest.

Yours sincerely,

The copy of Mike’s letter follows:

“Dear Bob,

I’ve just read your comments on NEON in the October issue of MacTutor (which takes a while to arrive on the newsstands here). I was amazed (and delighted) to see that there are still a number of folks using NEON - I thought I might be the only one! I have always thought that this was the most underrated development system for the Mac. A good product with potential to be a great one, but torpedoed by a marketing strategy which bordered on the nonexistent.

I also found no problem running on a Mac II - my personal finances don’t run to one, but I had a report that an application that I wrote in NEON worked OK on a Mac II. Like you I found I had to set a larger stack size than the default - unless I set at least 7K I got random crashes, usually during Standard File calls. Maybe this was the cause of the problems others ran into.

I also started to develop NEON, but thinking I was about the only user I took a more radical line than you - I completely reimplemented the nucleus from scratch. It now compiles subroutine-threaded code, with optimized native code for common sequences. The result is that applications run 4-5 times faster, and are about 10% smaller. The compilation speed is now over 1000 LPM. The whole assembler compiles in about a minute. I’ve also introduced quite a few improvements to the high-level code - at least I think they are improvements. Multiple inheritance, for example. I thought the changes were radical enough to justify a change of name, so now I’m calling it Mops (Michael’s Object-oriented Programming System!)

My understanding of copyright legalities is that the Mops nucleus, being completely new, would not violate Kriya’s copyright on NEON. That is unless a language, as such, is copyrightable, rather than the specific code which comprises a specific implementation. I think it probably isn’t copyrightable, but I’m not a lawyer. But since I haven’t changed all the high-level code, Kriya still definitely hold the copyright to that (unless you were able to persuade them to release it into the public domain - how did you get on?). But I’m quite happy to send Mops to anyone who has NEON, since that doesn’t involve legal problems. In any case, I don’t have the time or inclination to document Mops from scratch for someone unfamiliar with NEON. I will produce a file listing the changes between Mops and NEON, if anyone wants Mops. If I’m the copyright holder on the Mops nucleus, my only conditions are that it not be used for commercial purposes without my permission.

So if you or any other NEON users are interested in giving Mops a try, just write me, or better, send me a disk with some of your NEON developments on it, and I’ll send it back to you with Mops. Sorry for the horse and buggy method, but distance precludes electronic transfer. Perhaps you could do me a favor and post some of this letter on some appropriate bulletin board, in case anyone might be interested.

Of course, if your NEON code makes much use of implementation details of NEON, like expecting certain things to be at CFAs or whatever, it won’t work under Mops without rewriting. Also, I took a different line on floating point. Actually, this was only experimental since I don’t need FP in my work - it’s all string handling. NEON FP is very implementation-dependent, so a fair bit of work would have to be done to get existing FP code to work under Mops. But it certainly could be done.

Apart from that, the main differences from NEON are in the direction of improving the error checking, and also in bringing the underlying Forth more in line with Forth 83 and the forthcoming ANSI standard. Most of the necessary changes to NEON code can be done semi-mechanically. I found the most troublesome was the change in boolean “true” from 1 to -1. If you don’t often do arithmetic on booleans, you won’t have much trouble.

I’ve also tried to make Mops 32-bit clean, and it seems to work OK on new Macs. At least it worked on a friend’s SE/30 (which cheered me up, and slightly compensated for the green colour of my face).

I haven’t had time yet to update all the various NEON utilities, or the classes that I don’t actually use. But I have updated all the standard stuff, implemented a modeless dialog class, and written a new decompiler/debugger. I also have code that will read and write formatted Microsoft Word documents, if that’s any use to anybody.

Looking forward to hearing from you,

I have checked back with Bob Loewenstein <rfl@tycho.yerkes. uchicago. edu>, who is sending me his NEON extensions for review. You’ll hear about it in one of the next columns. It also seems that Kriya is still planning to release NEON into the public domain, but that there were some legal problems that had to be fixed before they could actually go ahead with it. There may be a chance that NEON is actually public domain at the time you read this.

MacForth 4.0

It is a long time since I wrote something about MacForth; that was when version 3.53 came out. Mainly because of the closer machine-orientation of Mach2, and also due to my non-access to Compuserve’s MacForth forum, I preferred to write my Forth examples in Mach2. Also, we had hardly any feedback from the MacForth community in the form of articles we could publish. However, that does not mean that Creative Solutions and the MacForth users group have been asleep. Just now, a thick binder with the 4.0 manual and diskettes arrived here, and I’ll give you some of my impressions. I cannot possibly cover all the features here, because of sheer quantity; I’ll only talk about the important new improvements. The two changes that are most visible compared to the previous version (3.53) are the support of stand-alone code, and active elements, a sort of object orientation.

Stand-alone code

The great advantage of Mach2 over MacForth has always been that Mach2 created true 68000 code through its subroutine-threading mechanism. Therefore, it was easily possible to create stand-alone code resources in Forth, as we have seen in various examples of XCMDs, MDEFs, FKEYs and DRVRs. A token-threaded Forth like MacForth has much greater difficulties to create stand-alone code, because the Forth code needs an interpreter to be executed.

Fortunately, MacForth’s interpreter is very small, it consists of two instructions:

 MOVE.W (A2)+,D3
 JMP  0(A6,D3.W)

where A2 is the instruction pointer and A6 contains the base address of the MacForth kernel. The interpreter, the token-threaded Forth code, and some entry and exit code together will constitute a stand-alone code resource (SA) which can - for instance - be called by a toolbox routine which requires a filter procedure.

The process of creating SAs is very simple. Let’s assume you have a Forth definition that you want to convert into an SA, such as

: beeper 10 sysbeep ;

After loading the SA maker file (which contains all the source to the SA compiler), you simply type SA beeper. This initiates a rather complicated process, where the Forth definition is analyzed: a list is created of all tokens used, and the definitions of all these tokens copied into the target area. Of course, this process is recursive, because the copied tokens may call other tokens, etc. The token values of all definitions have to be changed during this process, because tokens are actually offset addresses from the base of the kernel, and all those offsets will have changed.

One can follow the compilation process when the debug option is on; a message will be printed out as each Forth word is copied into the target area and the token addresses are changed.

The stand-alone support code included with MacForth is very complete; examples are given for I/O (Forth I/O does not work from SAs, of course), floating point handling, and debugging. Other files contain code used for generating drivers/DAs and XCMDs/XFCNs. For the latter, interfaces are also given to the Hypercard XCMD callback routines.

A simple example of an XCMD which beeps a number of times depending on the first parameter is given below:

\ 1

: nbeep \ MacForth XCMD to beep n times
 paramCount 0= IF 0 return EXIT THEN
 0 locals| HANDLE |
 0 param -> handle
 handle lock.handle
 handle h@  \ convert to address
 xcmdPad ZeroToPas \ convert to Pascal string
 xcmdPad StrToNum  \ convert to a number
 0 do 10 sysbeep loop\ beep n times
 handle unlock.handle
 handle dispose.handle
 0 return
;

To convert this code into an XCMD, you simply type

--2

xcmd nbeep
2000 “ nbeep” store.xcmd

and the XCMD resource will be left in the “Resource Bag” file where all standard resources of MacForth are stored. Included on the source code disk is a Hypercard stack with which you can try out this XCMD.

Actels - a simple OOP system

MacForth 4.0 now includes support for a simple late-binding message-passing mechanism to objects, which CSI calls actels (short for active elements). Like classical objects, actels are data structures bundled with the procedures that operate on the data. When messages are passed to an actel, an appropriate procedure will be invoked, depending on the message; when no method corresponding to the message can be found, the message is passed on to the parent object in the hierarchy. So far, just like standard object-oriented behavior; the difference is that the behavior of an actel is much less defined initially than that of objects in other languages. For instance, classes as such don’t really exist; to create a new class of actels, you define a prototype, which should be copied and initialized when it receives the >>New message. It is up to you to define this behavior; there is no generic ‘object class’ for which some default behavior is implemented.

To make this more understandable, let me describe how the example on the MacForth disk implements matrix actels. First, a data structure is defined, which contains a 16 byte header and cells that hold the row and column size; no space for storing the data is defined yet:

\ 3

structure MATRIXHEADER
 minElemHeaderSize + \ generic header info
 short: +#ROWS   \ number of rows
 short: +#COLS   \ number of columns
 0 bytes: +DATA  \ variable amount of data
structure.end

The first four bytes of the header contain a jump to the method selector corresponding to the actel. The data following the header are the instance variables; the length of the instance variable list is stored in the header. You are responsible for initializing the actel prototype, which means storing the jump to the method selector and the instance variable length in the header, and initializing the other variables.

Messages to actels and the corresponding methods can be defined very simply: each message corresponds to a number, but normally you don’t need to know that number once the message is defined. For example, the messages to the matrix actels are defined as follows:

2000 message >>RC@
2001 message >>RC!
2002 message >>M+
2003 message >>M*

As a convention, names of messages start with “>>”.

Selectors for matrix actels would look somewhat like the following. First, we define the low-level methods that respond to some default messages, their numbers start with 1:

\ 4

1 selector: LowWMatrix    ElemPanic\ no parent
 WMatNew\ >>New
 to.heap\ >>Discard 
 drop   \ >>Empty 
 inherited  \ >>Room 
;selector

The first element in the list is the name of the method that is invoked when the message can’t be handled by this actel. Since there is no parent object, we use the generic ElemPanic procedure, which displays an error message. Messages number 1-4 are predefined by MacForth, but their behavior is not; the next four elements in the selector list are Forth words that define the behavior.

Similarly, we define a selector list for our newly defined messages to matrix actels, their numbers start with 2000:

\ 6

2000 selector: HighWMatrix
 LowWMatrix \ parent
 RCW@  RCW!  M+   M*
 \ see message descriptions above
;selector

The header of our matrix actel prototype must then contain a jump to the highest-level selector, HighWMatrix.

When a new instance of an actel is created, the header and instance variables are copied into a new heap object of the appropriate size. That heap object can then be re-sized, for instance to accommodate more rows and columns.

Methods are invoked on actels by pushing the actel’s handle on the stack and calling the message: for instance a b >>M+ for adding the matrices a and b.

You see from this brief overview that actels in MacForth Plus provide a very general way of implementing object-oriented behavior; the ‘disadvantage’ is that much of the details are left to the programmer, but I see this rather as a great advantage since it offers great flexibility in defining the behavior of objects. Very Forth-like, because a. it restricts the programmer in the least possible way and b. of course, it will allow you to do the most stupid things without great difficulty. But then, if you want a safety net, you can always resort to C++

Speed

Speed of code execution, as you recall, has always been one of my major concerns in judging development systems on the Mac. Therefore, the overview of MacForth would not be complete without giving you the results of some benchmarks. The good news is that MacForth now comes close to Mach2 in execution speed, the bad news of course that it is not quite there yet. You have to pay a price for token-threaded code. Nevertheless, the Sieve benchmark - with local variables and some optimized words - executes in 2.9 seconds on a MacIIx. Mach2, with its built-in words, takes 2.0 seconds. If one optimizes the sequences i c@ , 0 i c! , and i + in Mach2 by defining the code words ic@ , 0ic! , and i+ like in MacForth, Mach2 will take 1.95 seconds - the built-in optimization of Mach2 is quite good. Under MacForth, the sieve version without the optimized words executes in 3.6 seconds.

Floating point execution speed, which has been a major concern for the previous version of MacForth, has now been greatly improved; a 68881 version of the floating point words is included, and now there is almost no speed difference for executing standard floating point operations between Mach2 and MacForth. The old MacForth floating point routines which use the SANE package, however, still have quite some overhead for empty loops.

Compilation speed in MacForth Plus is very fast; I see a noticeable difference to the old version, and the documentation says there is an increase in speed by a factor of 3-5. Part of this improvement is because of changes in the vocabulary structure; vocabularies have a completely new internal structure, they are hash-coded and are actel, thus inserting a new word into a vocabulary means sending a message to it. These vocabularies are much smarter than the previous ones; for instance, they automatically resize themselves when they run out of space.

Conclusion

There are many more improvements in the new MacForth, but these are the major ones. Stand-alone code, a domain previously only covered by Mach2 Forth, can now be created by MacForth as well, and there are very efficient tools provided with the system to help programming such stand-alone code. Together with the object-orientation mechanism provided by the actels, and the great variety of programming aids and examples provided, MacForth’s new version provides one of the best compromises between versatility, speed of development and speed of execution that I know of in a Macintosh development system. Of course, any Forth programmer will affirm that only a Forth development system will be able to offer such a good balance between flexibility, simplicity and speed, (everybody here knows that’s my opinion, anyway), but that’s another story. My personal preference for ‘close-to-the-toolbox’ programming still remains Mach2, also because of its good assembler, but for Forth programmers who prefer a very extensive set of words and tools that provide a very high-level access to the Macintosh, MacForth is the system of choice. CSI’s user support, through upgrades, newsletters, and the Compuserve Forth forum, is excellent, and if the examples on the MacForth disks don’t satisfy you, you can find orders of magnitude more on Compuserve. If only Palo Alto Shipping’s Mach2 support on GEnie were a little more active lately, one hasn’t heard much of them.

Anyway, you see, the Forth is still with us.