Mach2 and InterNet

By Jörg Langowski, MacTutor Editorial Board

Mach2 - any news?

Mach2 Forth is still alive and well - at least on my Macintosh. For me, it remains the ideal vehicle to test new system features, new managers, etc. because of the incremental compilation (Forth !), its well-designed assembler and the possibility to program and debug very ‘close to the machine’ - at each stage, you can very easily keep track of what your program is actually doing in detail.

Just to prove that others are using Mach2 to develop applications of very important size, here’s a letter from Switzerland:

“Hello Jörg,

Brief Intro:

FORTH (MacFORTH and MACH1&2) user since the advent of the Mac 128K. Project manager and coauthor of a 50k line MACH2 ECG analysis program, now maintaining and reworking the whole program alone.

My Question: Do you know anything about the state of Palo Alto Shipping Company. I was in Palo Alto in November 89 for one week and tried to reach Lori Chavez. I presume the company died because only the answering machine responded to my calls, nobody called back despite the messages I left, nobody was at the company’s office and they even stopped advertising in MacTutor. I would be more than happy to hear that they are still up and running, but if my assumptions are correct, the further development of our product will be severely crippled when System 7.0 arrives.

Thank you for responding

[I have noticed similar things. There is an answering machine at PASC’s number, (415) 688-1111, but I haven’t been successful at reaching them, either. Also, there have been almost no more new messages on the Mach2 round table on GEnie for months, and none at all from PASC. I hope this is not going to end like the NEON story that after a while we’ll be in a position to try and get Mach2 into the public domain, because the company has abandoned the product. It would be a real shame, given the qualities of Mach2. I am sorry I cannot give you better news for now. Does anyone else out there know more? Please write or drop us a link.]

One can learn a lot about Macintosh code segmentation in general - not only applied to Mach2 - from a note that reached me from Norway:

“Dear Jörg,

Included is a ‘Bug report’ and a short talk on NEW-SEGMENT, things I have had to discover the hard way. Can this be of any use as a MACH2-Forth revival for MacTutor? Or has the world of C++++, MWP This, MWP That, Lisp, Serieus, Hypercard, Pascal Supercharge , Oooops, done away with all the “Mac”-Forthers?

Several months ago, I have written to Palo Alto Shipping reporting about the above named bug and hoping to receive an answer on a Why-does this happen? question. I think, they are busy MacApping...

I believe Mach2 Forth is a viable alternative on the Macintosh, and I would hate to see it disappear. I believe there is still a lot of scope for improvement.

Conrad Weyns

Conrad’s note on code segmentation follows:

Use of NEW-SEGMENT in Mach2

Important considerations that are NOT documented anywhere in the Mach2 manual.

Developing any reasonably sized application in Mach2 Forth, will sooner or later bring one to the issue of CODE segmentation. Not only because of the 32K limit in relative addressing, a necessity for relocatable code [on 68000 machines, not on the 68020 and higher - jl], but also because dividing the application into logical entities that use memory only when needed seems a good practice.

Code segmentation becomes a key issue and is in fact very much in tune with the Forth idea of “Factoring”.

In the Mach2 manual we are told that we can use NEW-SEGMENT any number of times and need simply to precede the words that will be referenced in later segments with GLOBAL.

Jörg Langowski suggested in the Jan89 issue of MacTutor to gather all VARIABLEs, MACH words and Compiler utilities in one segment, as this segment is only needed for compilation and can consequently be removed from the final TURNKEYed application [CONSTANT in MACH2 does not use any code (dictionary) space, only vocabulary space which in the turnkey gets discarded anyway. So the above represents no winning for CONSTANTs. This is very different in other Forth implementations!].

This is a good idea, though easier said than done. It can be very hard to know so early in the development process just how many variables you are going to need and WHAT good descriptive name to give them. Remember the issue of this talk is Segmentation and Reasonably Large Program Size (e.g. 10-15 user segments and a total size close to 200K bytes).

Having to back up and recompile 10 segments just because you need a new variable is a pain in the and after all, we are using Forth because we love it, and sequential loading-testing-debugging of small quantities is the name of the game. If you would have to recompile everything every time, why use Forth [Yeah, why? Might as well use C++ then jl].

Mach2 manual p.61: GLOBAL will add the next word defined to a linked list of words which should be given jump table entries the next time a NEW-SEGMENT is made. Use GLOBAL sparingly, it adds 8 bytes to the program size for each definition (a jump table entry is 8 bytes long).

In reality, NEW-SEGMENT adds the 8 byte jump table entry to the MACH2 CODE 0 resource, which is the jump table. It adds to that resource which is already over 4K large because it contains entries for all existing predefined MACH2 words. This is why the MACH2 application ‘grows’ after each NEW-SEGMENT.

TURNKEY is the word that actually copies the whole of the MACH2 CODE 0 rsrc into your application.

If you are no aware of this and, in changing and recompiling from an earlier segment, you go on using the same MACH2 copy, you’ll end up with a huge CODE 0 resource because NEW-SEGMENT/GLOBAL will just go on adding on top of the existing jump table. When my turnkeys started to behave really weird and I finally started to look into this topic, my CODE 0 rsrc was over 34K bytes long! It should have been a mere 6K!

Let’s say you have about 10 segments of your own and you need to change and recompile the 6th one. Without having taken special steps on the way up, you will have to recompile and re-segment everything with a fresh copy of MACH2: Launch Mach2 -> load all seg#1 files -> NEW-SEGMENT seg#1 -> wait for it to finish returning to the Finder -> double-click on the new segment-document that also launches MACH2 -> wait for it to come up -> load all seg#2 files -> etc. etc !

What can we do? One thing is to make a copy of the MACH2 application after a NEW-SEGMENT, give it a descriptive name and keep it hidden in some folder close to the files belonging to that segment. This way we won’t have to recompile everything, only everything after the segment we had to back up to (6th in the above example). A large hard disk partition will be needed, and some Finder housekeeping will have to be done between segmentations but it sure saves time in the long run. Note that if you need to recompile from segment #n you’ll have to double-click the segment #n-1 document and its associated Mach2 copy!

[Another method would be just to ignore the ever-growing CODE0 resource during development, and always work with the same copy of Mach2. Then, in the end, when the final application is built, one recompiles everything from scratch using a fresh copy of Mach2 - jl]

Suggestion for a new Mach2 version, if ever:

• Keep the new global jump-table entries in the new segment document and rebuild it somehow on startup. This way, a single copy of the Mach2 application would suffice, and double-clicking a previous segment document will restore everything to where it should be.

• Design a shell application that will enable automatic sequential loading of all segments up to and including the Turnkey.

What else can we do to facilitate the task of changing and recompiling earlier segments? Consider the following: We have a new 32K segment and the very first definition is:

 GLOBAL
 CODE MyWord ( - )
 0 W, ( Placeholder for a JSR )
 0 W, ( Placeholder for its displacement)
 RTS
 END-CODE

Then go on with all the sub-words:

 : Do This    ;
 : Do That    ;
 : Do WhateverYouWant    ;

Then, the main word:

 : (MyWord)
 DoThis DoThat DoWhateverYouWant ;

and patch the main word before segmenting:

 ‘ MyWord ‘ (MyWord) PatchMe
 NEW-SEGMENT MyWordSeg

PatchMe is defined earlier in a disposable segment, and is an immediate word that simply calculates the correct positive offset for a JSR instruction which it emplaces in the body of MyWord.

 : PatchMe { ptr1 ptr2 | offset }
 ptr2 ptr1 - 2- -> offset
 $4EBA ptr1 w! ( JSR )
 offset ptr1 2+ w! ; immediate

Now we can “grow” or “shrink” the amount of code in this segment as needed, recompile with the relevant Mach2 duplicate, and copy the new MyWordSeg out of the produced segment document right into the turnkeyed application, replacing the existing segment. (Do not meddle with the IDs!). We have made change in an earlier segment and did not need to recompile everything else! A very substantial time saving.

[This works with any segment where all the GLOBAL definitions are kept in the same position at the beginning of the segment, and changes are made only at the end of the code. Of course, the patch is needed, since you have to access the code at the end somehow - jl]

Certain things to watch out for:

You can have as many GLOBAL entries at the start of the segment as you need, but you must not move or change them. After all they are only a means of pointing to the real thing later on.

You should not define new VARIABLEs: If you had not used VARIABLE at all after this particular segment, the new variable will not have a reserved address in the turnkey, and if you have used VARIABLEs in later segments the new one will now be at the same address as the next one defined in a later segment. (This was the most tricky bug that I ever dealt with, so gentle and unpredictable!)

[Remember that Mach2 reserves variables as offset from A5, and assigns space to variables in the order of their definition - jl]

If you do need new variable space you will have to put it into the code area, with CREATE or HEADER. Also, little-used code probably references little-used variables, so why occupy global variable space? [Well, there is a reason. Code space should not be used for storing things - in memory management systems to come, code and data areas might have to be separate. I don’t know when it will ever come into effect, but I think Apple plans changes in that direction - jl]

You can change everything else. If your change now happens to call a Mach2 segment that has never been referenced before, e.g. the TALKING vocabulary, you’ll also have to paste that segment into your turnkey.

Other considerations about NEW-SEGMENT: NEW-SEGMENT marks the last segment (the one it is actually creating) as locked. All previous segments which are copied into the new segment document lose their attributes as well as any names you might have given them with ResEdit. Thus, remember you set the correct resource attributes when recompiling a segment and pasting it into an existing application, e.g. segments that are purgeable must be marked as such. After all, the primary reason for segmenting code is to be able to free space by unloading segments, and UnloadSeg only unlocks the resource, making it float in the heap.

Is there room for my CODE resource? Loading CODE resources into memory and updating the jump table entries is done by the Mac’s segment loader, who is unforgiving about lack of memory. So you should test to see if there is enough memory available before calling a Global routine in another segment and give the user some feedback if there is a problem. Something like:

 CodeID CheckCode?
 IF DoIt ( GLOBAL def in other seg )
 [‘] DoIt CALL UnLoadSeg
 ELSE SorryGetaBiggerMac
 THEN

I have CheckCode? in a small segment of its own, preloaded and locked together with all necessary DLOG and other resources, always resident in memory. No point to have a fancy user interface with lots of resources if you are out of memory when you need them to alert the user!

What other use can we make of this?

 GLOBAL
 CODE MyWord ( x\y - )
 DROP
 DROP
 RTS
 END-CODE

• Why write it like this? (DROP compiles a 2 byte instruction). The word expects two parameters on the stack and just drops them. I don’t know yet how to write MyWord, but I know I will need it later on and know it will be passed two parameters on the stack. I want to get back to this later, but need to reference it now in Menu or Control handlers. So I simply compile the above definition in a fresh segment, execute NEW-SEGMENT and go on. Later I will write the actual code for MyWord, load, segment and paste it into the turnkey.

• The opposite is relevant as well. I have a segment that’s used only as a debugging aid. The main and only GLOBAL word is called ?CheckDepth and is used in places where I know the parameter stack should be empty; if not, it will show a dialog with the depth of the stack, the TOS (top of stack) item in both decimal and hexadecimal and the string whose address was on the TOS when called. This has proven to be of immense help in getting things to work properly and gaining a better understanding of the Mach2 system. Especially negative depth errors can be very hard to discover.

Sooner or later I will not need ?CheckDepth anymore, then I will simply replace that segment with:

 GLOBAL
 CODE ?CheckDepth ( $Adr - )
 DROP
 RTS
 END-CODE
 NEW-SEGMENT EmptyCheckDepth

I like to think of the above as the Mac alternative to the Forth problem of forward referencing. Variable @ EXECUTE is a viable Forth method but with the disadvantage of having to be initialized at run time.

Mach2 is the only Forth implementation that offers you “CODE segmentation à la Mac”.

Do we still know our machines?

I hope that Conrad’s letter has shown you that there are very interesting things to discover about Mac programming by using a well-designed simple development system like Mach2. I think very few programmers do actually know - or care - what their compiled code exactly does on the machine. For example, I am not fully aware of the way object-orientation is implemented in Object Pascal or C++; but looking at systems like NEON, Wayne Joerding’s Object Forth or the Actels in MacForth one can start to understand how these things are done in detail. For some other things, machine-level details matter a lot - developing fast algorithms or systems programming on the driver level are some examples.

In general, the Macintosh has evolved from a machine that one person could almost fully understand in detail - with some effort -, to a system whose complexity matches (or even exceeds) that of mini-mainframes. Gives great comfort to the user, and to the programmers developing application software that follows Apple’s standards to the letter. For the quality of the software developed on the Mac, this is certainly a big advantage. But I can’t help feeling uneasy about developer’s guidelines, however well thought out, that don’t explain to me why on earth I should do this and avoid that. In a way, a very hierarchical system: you receive orders from somewhere that should be followed, or else - as the famous quote says, Forth is for anarchists.

Personally I would like to see the Mac evolve in a way that I can still understand every little thing in principle, if I take the time and the effort. The current way things develop is certainly going into the opposite direction, with MacApp’s “don’t call us, we call you”. Whether this is the right or the wrong direction, I don’t dare to say.

Mac and the Networks

An Applelink message that came in recently made me realize that we should provide you with some more information how to use computer networks and how they relate to the Macintosh world. There are quite a few people who don’t realize the extent of services that can be obtained through public-access computer networks, ranging from E-mail over discussion lists to public domain software depositories.

From: D0999 Acropolis SW, Kim Hunter,PRT

“MacTutor, April 90 your article mentions Internet. What is Internet? I’ve heard much mention of it, but never how to get on. How does one find out about how to obtain an Internet account?”

Internet is just one of a world-wide system of interconnected computer networks for sending electronic mail and transferring files. The main users of this network are large institutions such as government organizations, universities, research institutions and large companies. Internet addresses consist of several names separated by dots, such as apple.com for an obscure small computer company in the Bay Area. Other names might look like cunyvm.cuny.edu or tcgould.tn.cornell.edu for some machines at universities, for example. It really doesn’t matter what the different sub-fields of those addresses mean; I just wanted to show you the syntax. If you are a user of a computer that is connected to the Internet, you can send mail to other Internet addresses by using the mail services available on your machine. You can also - in some cases - remotely log on to other machines and transfer files between them. Such remote file access is also called ‘FTP’. Many machines on the internet offer guest accounts with FTP access from which one can download public-domain software or other information.

Now the practical question: how do you access those things, or send mail, if you don’t have access to a machine that is connected to the internet? Of course, you must have access to some computer network. Since most of the existing networks are connected to each other through gateways, there are actually many possibilities to exchange information across network borders.

For instance, there exists a gateway between Applelink and Internet. If you want to send mail to someone whose internet address might be, e.g., fred@myvax.xyz.com, you simply use the address:

fred@myvax.xyz.com@INTERNET#

in the To: field in the Applelink Send dialog, and send your mail message in the usual way, it will be forwarded. Other networks that are not part of Internet can be addressed using a syntax that looks similar. Our machine, for instance, is the node FREMBL51 on Bitnet, a global academic/research network, also known as EARN in Europe. My address there is langowski@frembl51. Through the Applelink/Internet gateway, you can send me mail at langowski@frembl51.bitnet@INTERNET#. I can reach Applelink users from Bitnet through gateways that interconnect Internet and Bitnet. The mail software is intelligent enough to allow me to simply type an Internet address, and the mail is sent to the correct gateway and forwarded. For instance, I can send mail to the MacTutor main offices at mactutor@applelink.apple.com. Thus, the Applelink system looks like one node on the Internet, with each Applelink address being one ‘user’ at that node. From my Applelink account, I can also send mail to myself by using the address

langowski.j@applelink.apple.com@INTERNET#.
 

This way, my mail will be sent right back to me!

Compuserve users, too, can exchange mail with Internet addresses; the Internet address of your friend whose Compuserve account is 76543,2109 would be

76543.2109@compuserve.com.

I have not found out whether one can access any of the public domain software depositories through Applelink. However, those of you with FTP access to Internet machines should be aware of two addresses which offer a large selection of Macintosh PD software and shareware:

sumex-aim@stanford.edu and wsmr-simtel20.army.mil.

Of course, Apple information, some software, tech notes etc. can also be found at apple.com.

Bitnet users can access the PD software archives through gateways: sending the message ‘get filexyz.abc’ to MACSERVE@PUCC or MACSERVE@IRLEARN will tell those machines to send you the file filexyz.abc from the Stanford archives, and if you just send the message ‘help’, they will send you an information file. The Simtel20-archives are accessible through the gateway LISTSERV@RPIESC, just send ‘help’ to get information.

This information is far from exhaustive. There are other freebie file depositories on many machines around the world, discussion groups on topics from ecology over science fiction to Macintoshes (of course) and Forth. If you need any more specific information, feel free to drop me a line at langowski@frembl51.bitnet or langowski.j (Applelink).

Next time I’ll give you some object orientation again - C++, of course, and we’ll take another look at the NEON scene, where exciting things happen.