TweetFollow Us on Twitter

Forth Decompiler
Volume Number:1
Issue Number:2
Column Tag:Forth Forum

“A Forth Decompiler”

By Joerg Langowski

“A Forth Decompiler”

Everyone of you Mac Forth users is familiar enough with Forth to know that it is a ‘threaded interpretive’ language. A Forth definition (as you type it into your machine) consists of a string of other, previously defined Forth words, and is compiled as a string of addresses that point to the definitions of these other words.

This makes for a rather fast interpretation of the resulting code. However, some of the very primitive words and those words whose execution is time-critical may also be defined in machine language. MacForth has devised a very elegant way to distinguish between Forth words defined from within Forth and machine code.

Structure of a Forth Definition

The object code of any Forth word, as it is compiled into the object dictionary, starts with at least one 16-bit word that is a meaningful executable 68000 machine language instruction. When the Forth word is executed, the interpreter simply jumps to this address. Forth definitions (colon definitions, constants, variables) now start with one of the 68000 TRAP instructions ($4E4X, where X can be anything from $0 to $F). The corresponding trap vector points to a routine which e.g. in the case of a colon definition gets the next 16-bit word and interprets it as a Forth token (converts it to an address and executes it), or - in the case of a variable - puts the address of the variable on the stack.

If the word is defined completely in machine language, the code is executed until a special JMP instruction transfers control to the next higher level (I’ll describe that later).

At this point I have to confess that I would not have come even this far if it had not been for two excellent routines that I found on a CALL-A.P.P.L.E. public domain disk. One of those - a Forth decompiler - is included below so that you can enjoy hacking into the Forth engine, the other one - a disassembler - was too long to be printed here.

An Example

Admittedly, part of the above sounds a bit dry and theoretical. Lets look at a simple example.

Assume you had defined the word TEST as follows:

: TEST DUP 2* SWAP DROP . ; 

The Forth compiler will then create a list of 16-bit words that looks like:

 $4E4F  (trap for colon definition)
 $0498  (Forth token for DUP    )
 $074E  (   “          “     “   2*     )
 $049C  (   “          “     “   SWAP)
 $00EC  (   “          “     “   DROP )
 $0EBE  (   “          “     “    .     )
 $0060      (   “          “     “  EXIT  )

Interpretation always ends at the EXIT token.

TOKENS

What are those ‘tokens’? They are the starting addresses of the Forth definitions that are offset by a constant that is contained in register A4 (probably to make the object code relocatable). There is a word in MacForth that converts a token to an address, TOKEN>ADDR. The token of a word is extracted from the vocabulary by the Forth word, FIND. Therefore, you will get the starting address of the example above by executing

FIND TEST TOKEN>ADDR .

The address that you’ll see displayed, of course, depends on how much object code you have already in your system. Let’s call this number TESTADDR. Then define the following word:

 :  TEST.DISP  7 0 DO I 2* TESTADDR ( insert your # here) + W@ . LOOP 
;

and execute TEST.DISP; you will see the list of words above.

This way you can decompile any Forth word that you find in the system. The decompiler is somewhat more convenient, of course; if you use the procedure above, you still have to convert the tokens into Forth words. This is done (for one token on the stack) by executing

 NFA ID.

This converts the token (not the pfa, as the Forth 1.1 manual says) into the name field address (NFA) and then displays the name of the word (ID.).

Machine language definitions

What if the definition is direct machine language code? Again, let us look at an example, the word SWAP. FIND SWAP TOKEN>ADDR gives (in my system) $5B60. At this address, however, we find code that does not start with a trap statement; it is a routine that does what we expect:

 202F 0004MOVE.L 4(A7),D0    /move item below top -> D0
 2F57 0004MOVE.L (A7),4(A7) /move top item one down
 2E80   MOVE.L D0,(A7)       /move D0 -> top of stack
 4ED4   JMP (A4)                  
/get next token

We see that indeed the two top stack items are exchanged. The last statement is the end of any machine language Forth definition. This jump to the address in A4 is what I briefly mentioned above. A4 contains the address of a routine that gets and executes the next Forth token from the object code (which A3 points to):

 MOVE (A3)+,D0 
/next object token -> D0
 BMI L1 
/is it neg. get address from token table
 JMP (A4,D0.W) 
/jump to start of definition (token + A4)
L1 MOVE(A4,D0.W),D1
/get address from table
 JMP (A4,D1.W) 
/and jump to start of definition

Hidden definitions

When you decompile (with the program below) the word SELECT.WINDOW, you’ll see something funny. It seems to be a regular Forth colon definition; however, the tokens displayed seem to have no name. Only ??? and the token numbers are displayed. These are tokens whose names have been deleted from the vocabulary, but their corresponding addresses (A4+token) point to valid definitions. The reason why CSI did this is probably to keep the vocabulary short and to make words inaccessible to users whose misuse could have a disasterous effect on the system. Anyway, the word SUBLEVEL in the definitions below will decompile and display any such ‘hidden’ code, if it is a colon definition. It will display nothing for machine code definitions, you have to disassemble them.

SELECT.WINDOW, with this tool, then becomes very clear. Its first level definition looks like:

: SELECT.WINDOW {2142} {1B0E}  ;

where the braces indicate those ‘no-name’ tokens. {2142} merely checks if the pointer on the stack is a valid window pointer, to keep the toolbox routine from crashing; decompile it with SUBLEVEL to see what it does exactly. {1B0E} is a 2-word machine code routine:

 $A91F  
/toolbox trap for SELECT.WINDOW
 JMP (A4) 
/and get next word. That’s all!

Listing 1: Forth decompiler
( DECOMPILER Blocks File -- Version 1.00 )        ( ADG - modif. 110384 
jl ) 
 DECOMP  ( -- )                                                 
Decompiles the definition of the next word in the input stream.  A line 
is displayed for each word in the definition. Each line begins with its 
relative code address in hex.  Next is the name of the word.  Finally, 
if the word has an in-line parameter, it is shown.  If the word is a 
branching word, the value is the target address.  If the parameter is 
a token, its name is shown.  If it is a string, the string is shown in 
double-quotes.  If it is a word or double-word, its hex value is followed 
by its decimal value.                            
#DECOMP in block 8 can also be loaded by those who wish to write a routine 
to pass tokens on the stack to be decompiled. For valid tokens, its output 
is identical to that of DECOMP .
 Written: 07/21/84   By: Alan D. Galumbeck  [70220,200]         
 NO RIGHTS RESERVED    NO RIGHTS RESERVED    NO RIGHTS RESERVED 

BASE @ DECIMAL   VARIABLE HIGH.PFA  16384 MINIMUM.OBJECT  2048 MINIMUM.VOCAB 
        : .DIGITS  ( n1\n2 --  | Types the low-order n2 digits of n1 
)  0 <# DO # LOOP #> TYPE ;          : SPACE.TO  ( n -- | Spaces to column 
n or 2 spaces if past n ) COL @ - 2 MAX SPACES ;               : DISP.WORD 
   ( pfa -- pfa+2 | Display a 16-bit parameter)     
  DUP W@ 4 .DIGITS 31 SPACE.TO DECIMAL DUP <W@ . HEX  DUP W@    
  NFA ?DUP IF 42 SPACE.TO ID. THEN  2+ ;
: DISP.DBL     ( pfa -- pfa+4 | Display a 32-bit parameter )    
  DUP @ DUP 8 .DIGITS 31 SPACE.TO DECIMAL . HEX 4+ ;  
: DISP.STRING  ( pfa -- pfa+len | Display a string parameter )  
  34 EMIT COUNT 2DUP TYPE 34 EMIT + =CELLS ;                    
: DISP.TARGET  ( base.pfa\pfa -- base.pfa\pfa+2 )               
  ( Display a branch target and save if it’s the highest )      
  DUP <W@ OVER + DUP HIGH.PFA @ > IF HIGH.PFA ! ELSE DROP THEN  
  2DUP SWAP - OVER <W@ + . 2+ ;                                 
: DISP.TOKEN   ( pfa -- pfa+2 | Display a token parameter)      
  DUP W@ NFA ?DUP  IF ID.  ELSE DUP W@ 4 .DIGITS  THEN  2+ ;    
: DISP.ADDR  ( pfa -- pfa+4 | Display an address parameter )    
  DUP @ NFA ?DUP  IF ID.  ELSE DUP @ NEXT.PTR + 8 .DIGITS THEN 4+ ;  
  
: SPECIAL.TOKENS ( base.pfa\pfa\token -- [base.pfa\next.pfa] or 
                                         [base.pfa\next.pfa\0] )
  ( Handle in-line parameters and terminating words )           
  CASE  TOKEN.FOR  EXIT       OF        0                  ENDOF
        TOKEN.FOR  (;CODE@)   OF        DISP.TOKEN 0       ENDOF
        TOKEN.FOR  COMPILE    OF        DISP.TOKEN         ENDOF
        TOKEN.FOR  0BRANCH    OF        DISP.TARGET        ENDOF
        TOKEN.FOR  BRANCH     OF        DISP.TARGET        ENDOF
        TOKEN.FOR  (OF)       OF        DISP.TARGET        ENDOF
        TOKEN.FOR  (LOOP)     OF        DISP.TARGET        ENDOF
        TOKEN.FOR  (+LOOP)    OF        DISP.TARGET        ENDOF
        TOKEN.FOR (MENU.SELECTION:) OF  DISP.TARGET        ENDOF
        TOKEN.FOR  ALIT       OF        DISP.ADDR          ENDOF
        TOKEN.FOR  WLIT       OF        DISP.WORD          ENDOF
        TOKEN.FOR  LIT        OF        DISP.DBL           ENDOF
        TOKEN.FOR  (.”)       OF        DISP.STRING        ENDOF
        TOKEN.FOR  ($LIT)     OF        DISP.STRING        ENDOF
        TOKEN.FOR  (ERROR”)   OF        DISP.STRING        ENDOF
        TOKEN.FOR  (ABORT”)   OF        DISP.STRING        ENDOF
        TOKEN.FOR  $ADDR      OF        DISP.STRING        ENDOF
        ( Insert the ones I’ve missed here. )                   
        0                     OF        2 - DISP.TOKEN     ENDOF
  ENDCASE ;  

: DECODE.TOKENS  ( pfa --  | Display the words starting at pfa )
  DUP HIGH.PFA ! DUP                                            
  BEGIN                                                         
    HEX 2DUP SWAP - CR 4 .R 2 SPACES DUP 2+ SWAP W@ DUP NFA ?DUP     
                            
      IF    ID.    ELSE .” ???” drop 0  THEN                         
        
      20 SPACE.TO SPECIAL.TOKENS ?DUP                           
      IF    FALSE  ELSE  DUP HIGH.PFA @ > THEN                       
        
  UNTIL                                                         
  2DROP ;                                                       

: .VALUE  ( n1\n2 --  | Display constants and UA variables )    
  HEX .DIGITS .”  hex  “ DECIMAL . .” decimal )”  ;             
                                                                
: DECODE.VECTOR  ( pfa\vector -- | Display definition type )    
  CASE                                                          
    11 OF .” User Area variable ( Offset = “ W@ DUP 4  .VALUE  ENDOF
    12 OF .” 16 bit constant ( Value = “ <W@ DUP 4 .VALUE  ENDOF
    13 OF .” 32 bit constant ( Value = “ @ DUP 8 .VALUE   ENDOF
    14 OF .” Variable, array, or string” DROP              ENDOF
    15 OF .” Colon definition” DECODE.TOKENS               ENDOF
    .” Unknown code type ( Vector = “ 2 .VALUE .” )”            
  ENDCASE ;                                                     
                                                                 
: CHK.CODE.TYPE  ( token -- [pfa\vector\true] or [false] |      
   Returns false for machine code definitions, true for others )
   TOKEN>ADDR DUP 2+ SWAP W@ DUP 16/ 1252 = 
   IF 15 AND TRUE ELSE 2DROP FALSE THEN ;                     
 ( Note: 1252 is the machine code for a 68000 TRAP instruction  
         divided by 16.  Vector is the low-order four bits of   
         the TRAP instruction. )                                
: sublevel chk.code.type if drop decode.tokens then ;           
                                                               
: DECOMP  ( --  | Decompile the next word in the input stream ) 
  GET.LINE.HEIGHT  GET.TEXTSIZE  BASE @  9 TEXTSIZE  10 LINE.HEIGHT  
                                  
  +FIND CR POCKET COUNT TYPE .”  -- “                           
     IF 
        IF .” IMMEDIATE “ THEN                                  
        CHK.CODE.TYPE IF DECODE.VECTOR ELSE .” Machine code definition” 
THEN                   
     ELSE .” Not in dictionary” THEN                            
  BASE !  TEXTSIZE  LINE.HEIGHT  CR ;                           
                                                                
: #DECOMP ( token --  | Decompile word whose token is supplied )
  BASE @  GET.LINE.HEIGHT  GET.TEXTSIZE  4 PICK DUP             
  9 TEXTSIZE  10 LINE.HEIGHT NFA ?DUP CR                        
     IF DUP ID.  .” -- “ C@ 128 AND                              
        IF .” IMMEDIATE “ THEN                                  
        CHK.CODE.TYPE  IF DECODE.VECTOR  ELSE .” Machine code definition” 
THEN                   
     ELSE HEX 4 .DIGITS .” -- Not a valid token” THEN           
  TEXTSIZE  LINE.HEIGHT  BASE !  DROP  CR ;                     


 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Live Home 3D Pro 3.6.2 - $49.99
Live Home 3D Pro is powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac, iPhone or iPad. It has every feature of Live Home 3D, plus some... Read more
RapidWeaver 8.2 - Create template-based...
RapidWeaver is a next-generation Web design application to help you easily create professional-looking Web sites in minutes. No knowledge of complex code is required, RapidWeaver will take care of... Read more
Opera 60.0.3255.109 - High-performance W...
Opera is a fast and secure browser trusted by millions of users. With the intuitive interface, Speed Dial and visual bookmarks for organizing favorite sites, news feature with fresh, relevant content... Read more
DEVONthink Pro 3.0beta2 - Knowledge base...
DEVONthink Pro is your essential assistant for today's world, where almost everything is digital. From shopping receipts to important research papers, your life often fills your hard drive in the... Read more
Tunnelblick 3.7.9 - GUI for OpenVPN.
Tunnelblick is a free, open source graphic user interface for OpenVPN on OS X. It provides easy control of OpenVPN client and/or server connections. It comes as a ready-to-use application with all... Read more
Carbon Copy Cloner 5.1.9 - Easy-to-use b...
Carbon Copy Cloner backups are better than ordinary backups. Suppose the unthinkable happens while you're under deadline to finish a project: your Mac is unresponsive and all you hear is an ominous,... Read more
Dropbox 73.4.118 - Cloud backup and sync...
Dropbox is an application that creates a special Finder folder that automatically syncs online and between your computers. It allows you to both backup files and keeps them up-to-date between systems... Read more
Postbox 6.1.18 - Powerful and flexible e...
Postbox is a new email application that helps you organize your work life and get stuff done. It has all the elegance and simplicity of Apple Mail, but with more power and flexibility to manage even... Read more
Wireshark 3.0.2 - Network protocol analy...
Wireshark is one of the world's foremost network protocol analyzers, and is the standard in many parts of the industry. It is the continuation of a project that started in 1998. Hundreds of... Read more
BetterTouchTool 2.856 - Customize multi-...
BetterTouchTool adds many new, fully customizable gestures to the Magic Mouse, Multi-Touch MacBook trackpad, and Magic Trackpad. These gestures are customizable: Magic Mouse: Pinch in / out (zoom... Read more

AFK Arena guide - Everything you need to...
Ok, so if you're like me, you've been playing (and sometimes waiting) your way through AFK Arena, only to learn there's a lot more to it than there appears on the surface. There's guilds, a PvP arena, and all sorts of other systems and game modes... | Read more »
Explore an epic fantasy world in MMORPG...
Webzen have just announced the official launch date for its stunning MMORPG ‘MU Origin 2’ which will arrive for iOS and Android on May 28th. It will be the second spinoff from the classic PC-based MU Online, and it looks to further refine the... | Read more »
Solar Explorer: New Dawn guide - Tips an...
Solar Explorer: New Dawn is a lunar lander game that really ratchets the intensity up to 11. With all of the asteroids flying around as you fly around at seemingly breakneck speeds, it can be easy to feel overwhelmed bythe whole thing. | Read more »
The Dalaran Heist - How Hearthstone...
I am someone who wrote Hearthstone off a while ago. It was hard not to try and stick with it. The game has incredible production values and a core of really great talent working on the game continuously to keep it feeling fresh and fun (full... | Read more »
Steam Link App - Everything You Need to...
Steam Link has finally released for iOS! That’s right, you can play your epic backlog of PC games on the go now. Well… sort of. While the Steam Link app was announced seemingly ages ago, it only got actual approval for release last night. Check out... | Read more »
Pre-register now for endless superhero r...
Talking Tom Hero Dash is set to take the ever-popular Talking Tom and Friends franchise in a brand new direction as it opens pre-registration to players worldwide. Not only does it promise to be a beautifully rendered, fast-paced, action-packed... | Read more »
AFK Arena - Guild Wars guide
Ok, so if you're like me, you've been playing (and sometimes waiting) your way through AFK Arena, only to learn there's a lot more to it than there appears on the surface. There's guilds, a PvP arena, and all sorts of other systems and game modes... | Read more »
Superhero-themed Talking Tom Hero Dash i...
One of the exciting releases that we’re looking forward to is Talking Tom Hero Dash, an upcoming superhero-themed runner created by Outfit7. This new game is an action-packed endless runner that takes you on an epic adventure to assemble the... | Read more »
Kingdom Rush Vengeance Update Guide 2 -...
Kingdom Rush: Vengeance just got updated once again to add more content to the game. This addition, called The Frozen Nightmare, adds three new levels, five new enemies, two new heroes, and some new achievements. | Read more »
Save the world with SCIENCE in the upcom...
Previous versions of space colonization game TerraGenesis encouraged you to explore the galaxy and settle its planets. The eagerly-awaited 5.0 update will try to smash them to bits. Yep, with a new "world killers" setting, you can unleash... | Read more »

Price Scanner via MacPrices.net

12″ 1.2GHz MacBooks on sale for $999, $300 of...
Amazon has current-generation 12″ 1.2GHz Retina MacBooks on sale for $300 off Apple’s MSRP. Shipping is free: 12″ 1.2GHz Space Gray MacBook: $999.99 $300 off MSRP 12″ 1.2GHz Silver MacBook: $999.99 $... Read more
Here’s how to save $200 on Apple’s new 8-Core...
Apple has released details of their Education discount associated with the new 2019 15″ 6-Core and 8-Core MacBook Pros. Take $200 off the price of the new 8-Core model (now $2599) and $150 off the 15... Read more
Price drops! 2018 15″ 2.2GHz 6-Core MacBook P...
Amazon has dropped prices on clearance 2018 15″ 2.2GHz 6-Core Touch Bar MacBook Pros by $300 with models now available for $2099. These are the same models sold by Apple in their retail and online... Read more
Apple drops prices on 2018 13″ 2.3GHz Quad-Co...
Apple has dropped prices on Certified Refurbished 2018 13″ 2.3GHz 4-Core Touch Bar MacBook Pros with prices now starting at $1489. Apple’s one-year warranty is included, shipping is free, and each... Read more
Apple drops prices on 2018 Certified Refurbis...
Apple has dropped prices on clearance 2018 15″ 6-Core Touch Bar MacBook Pro, Certified Refurbished, with models available starting at only $1999. Each model features a new outer case, shipping is... Read more
Price drops! Clearance 2018 13″ Quad Core Mac...
Amazon has dropped prices on 2018 13″ Apple Quad-Core MacBook Pros with models now available for $250 off original MSRP. Shipping is free. Select Amazon as the seller, rather than a third-party, to... Read more
How Much Is ‘Solace’ Of Mind Worth When Buyin...
COMMENTARY: 05.22.19- Smartphone cases give us peace of mind by providing ample protection for such a fragile gadget and the sky’s the limit as far as choices go with a plethora of brands, styles,... Read more
Get a 13″ Touch Bar MacBook Pro for the lowes...
Apple has Certified Refurbished 2017 13″ 3.1GHz Dual-Core i5 Touch Bar MacBook Pros available starting at $1439, ranging up to $390 off original MSRP. Each MacBook features a new outer case, shipping... Read more
Apple adds new 15″ 8-Core MacBook Pro to line...
Apple has added a new 15″ MacBook Pro to its lineup featuring a 9th generation 2.3GHz 8-Core Intel i9 processor, 16GB of RAM, a 512GB SSD, and a Radeon Pro 560X with 4GB of GDDR5 memory for $2799.... Read more
21″ 2.3GHz iMac available for $999 at B&H...
B&H Photo has the 2018 21″ 2.3GHz Apple iMac on sale for $100 off MSRP. This is the same model offering by Apple in their retail and online stores. Shipping is free: – 21″ 2.3GHz iMac (MMQA2LL/A... Read more

Jobs Board

Best Buy *Apple* Computing Master - Best Bu...
**690427BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000860-Charlottesville-Store **Job Description:** **What Read more
*Apple* Mobile Master - Best Buy (United Sta...
**696430BR** **Job Title:** Apple Mobile Master **Job Category:** Store Associates **Location Number:** 001012-Bismarck-Store **Job Description:** **What does a Best Read more
Manager - *Apple* Team - SHI International...
…opportunity available in the Hardware & Advanced Solutions Department as the Manager of the Apple Team The Manager must be familiar with all aspects of Apple Read more
Best Buy *Apple* Computing Master - Best Bu...
**696375BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000203-North Austin-Store **Job Description:** **What does a Read more
Geek Squad *Apple* Master Consultation Agen...
**696286BR** **Job Title:** Geek Squad Apple Master Consultation Agent **Job Category:** Services/Installation/Repair **Location Number:** 000172-Rivergate-Store Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.