TweetFollow Us on Twitter

July 02 Section 7

Volume Number: 18 (2002)
Issue Number: 07
Column Tag: Section 7

by Rich Morin

More Basic Commands

and a first look at “shell scripts”

Previous “Section 7” columns have looked at several BSD commands, but they haven’t said much about the ways in which commands get performed. Let’s do that now, starting with the Terminal application. Each time you start up a new Terminal window, Mac OS X has to start up a program to talk to it. The program it starts is a “command interpreter”, most often referred to as a “shell”.

Whenever a Terminal window is active, your keyboard is tied to the “standard input” of the program that is currently running “in” the window. Similarly, the window’s text area displays the program ‘s “standard output”. If the shell starts up another program (such as ls), the shell must “loan” these I/O connections to the other program. In short, a Terminal window is always connected to some program; most often, this will be the shell.

Each shell runs as a separate process, so the “state” that gets set in one “shell session” is not shared with other shell sessions. Thus, if you use cd to change the “current directory” in one Teminal window, the shell sessions in your other windows will not be affected.

Because cd affects information that the shell must maintain (and hand off to any programs it starts up), cd is actually “built in” to the shell itself. Most commands, however, are not built into the shell. Instead, they are implemented as:

  • aliases or shell functions – These are sequences of commands, kept in the shell’s memory and interpreted upon demand.
  • shell scripts – These are sequences of commands, kept in separate files and interpreted upon demand. Sometimes the shell itself will interpret the script; other times, as discussed below, it will start up a separate program to do so.
  • executable binaries – These are separate programs, kept in separate files and executed upon demand. To run them, the shell must fork(2), then exec(3) the relevant file.

In the case of built-ins, aliases, and shell functions, the shell “knows” what needs to be done. In the case of shell scripts and executable binaries, however, it must find the requested file, determine its nature, and proceed accordingly.

If a command is not a built-in, an alias, or a shell function, the shell looks for an executable file of that name in one of several directories. Because the shell may cache the contents of these directories in an internal data structure, some shells require the user to run a special command (“rehash”) when a new command is added. The list of these directories, called the “search path”, is contained in the $path variable:

[localhost:~] rdm% echo $path
/Users/rdm/bin/powerpc-apple-darwin /Users/rdm/bin ...

The which command can be used to predict the shells’ behavior for a given command:

[localhost:~] rdm% which cwd l 
cwd:     aliased to echo $cwd
l:       aliased to ls -lg
[localhost:~] rdm% which cd echo setenv 
cd: shell built-in command.
echo: shell built-in command.
setenv: shell built-in command.
[localhost:~] rdm% which apropos keytool
/usr/bin/apropos
/usr/bin/keytool

Finding and executing files (e.g., /usr/bin/apropos) takes time, but it allows us to add new commands quite trivially. Imagine how awkward and constraining it would be if each new command had to be linked into the shell!

Now that we have the full path names of the commands, we can examine them a bit:

[localhost:~] rdm% cd /usr/bin
[localhost:/usr/bin] rdm% file apropos keytool
apropos: Mach-O executable ppc
keytool: Bourne shell script text

As file informs us, /usr/bin/apropos is an executable binary file for the PowerPC, encoded in “Mach-O” format. /usr/bin/keytool, in contrast, is a text file which must be run interpretively by sh (the “Bourne shell”) as a “shell script”. But wait a second; how do file (and the shell) know that? Maybe it’s time to take a look at /usr/bin/keytool:

[localhost:/usr/bin] rdm% cat keytool
#!/bin/sh
#
# Shell wrapper to launch keytool.
#
...

The first two characters of the file (“#!”) tell the user’s shell that this is a script. The rest of the line gives the full path name (“/bin/sh”) of the program which must be used to interpret the script. Because each script can declare its own language (e.g., Bourne shell, C shell, Perl), new scripting languages can be added at any time.

If a script starts with a sharp sign (“#”), but not the “#!” sequence, it is treated as a C shell script. If a script begins with any other character, it is treated as a Bourne shell script. Finally, you may encounter scripts that start like “#!/usr/bin/env perl“. This tells the system to run /usr/bin/env, which will then go off and find the “appropriate” copy of perl.

The same header information is used by file to determine the type of a given file. To see file’s list of “magic numbers” (and interpretations), let’s look at /etc/magic:

[localhost:/usr/bin] rdm% cd /etc
[localhost:/etc] rdm% wc -l magic
    3837 /etc/magic
[localhost:/etc] rdm% grep ‘shell script’ magic
0  string  #!/bin/sh     Bourne shell script text
...

“wc -l” (word count, in line mode) tells us that there are about 4K lines in /etc/magic, so we don’t try to list it. Instead, we use grep (global regular expression print) to display lines that contain the string “shell script”.

As discussed last month, most of BSD’s “system metadata” (e.g., control files, log files) is kept in ASCII format. This works well with BSD’s wide range of text processing tools, allowing us to extract and process information in a very flexible manner.

If /etc/magic had been encoded in some proprietary, binary format (e.g., Microsoft Word), the exercise above would have required us to start up a particular application. And, if the application wasn’t able to do what we wanted, we’d have been stymied.

Scripting languages

BSD provides a variety of scripting languages. You are free to use any, all, or none of them, as you prefer. Here are some (biased :-) notes that may help you in making your selections:

  • The Bourne shell (sh) is not particularly convenient as an interactive command interpreter, but it works well as a programming language. The Korn and Z shells (ksh, zsh) are upwardly-compatible versions of the Bourne shell. The former is not supplied as part of Mac OS X, but the latter is and may deserve a look.
  • The Bourne-Again shell (bash) is upwardly compatible with the Bourne shell, but it also borrows a variety of features from other shells. It is the default shell for most Linux distributions, but it is not supplied as part of Mac OS X.
  • The C shell (csh) is very convenient as an interactive command interpreter, but it loses badly as a programming language due to its lack of syntax recognition. For example, multi-line commands always require backslashes at the end of non-terminal lines, even when the shell should be able to recognize that the command isn’t finished. Most BSD systems, including Mac OS X, actually provide tcsh, rather than csh. As the tcsh man page says:
    “tcsh is an enhanced, but completely compatible, version of csh(1). It is a command language interpreter, usable both as an interactive login shell and as a shell-script command processor. It includes a command-line editor, programmable word completion, spelling correction, a history mechanism, job control, and a C-like syntax.”
  • Awk (awk) is a convenient and simple scripting language, suitable for simple formatting and calculation tasks.
  • If things start to get too complex for Awk or the shell to handle, you should probably turn to Perl (perl). Perl is an extremely powerful language, with a large and active user community.
  • Python and Ruby are object-oriented scripting languages with small but vocal followings. Neither language is supplied as part of Mac OS X, but (like the other languages mentioned above), they are readily available for you to download and install.
  • Finally, if you’re a Tcl fan, you might want to try out tclsh (“a shell-like application that reads Tcl commands“).

Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn’t writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Top Mobile Game Discounts
Every day, we pick out a curated list of the best mobile discounts on the App Store and post them here. This list won't be comprehensive, but it every game on it is recommended. Feel free to check out the coverage we did on them in the links... | Read more »
Price of Glory unleashes its 1.4 Alpha u...
As much as we all probably dislike Maths as a subject, we do have to hand it to geometry for giving us the good old Hexgrid, home of some of the best strategy games. One such example, Price of Glory, has dropped its 1.4 Alpha update, stocked full... | Read more »
The SLC 2025 kicks off this month to cro...
Ever since the Solo Leveling: Arise Championship 2025 was announced, I have been looking forward to it. The promotional clip they released a month or two back showed crowds going absolutely nuts for the previous competitions, so imagine the... | Read more »
Dive into some early Magicpunk fun as Cr...
Excellent news for fans of steampunk and magic; the Precursor Test for Magicpunk MMORPG Crystal of Atlan opens today. This rather fancy way of saying beta test will remain open until March 5th and is available for PC - boo - and Android devices -... | Read more »
Prepare to get your mind melted as Evang...
If you are a fan of sci-fi shooters and incredibly weird, mind-bending anime series, then you are in for a treat, as Goddess of Victory: Nikke is gearing up for its second collaboration with Evangelion. We were also treated to an upcoming... | Read more »
Square Enix gives with one hand and slap...
We have something of a mixed bag coming over from Square Enix HQ today. Two of their mobile games are revelling in life with new events keeping them alive, whilst another has been thrown onto the ever-growing discard pile Square is building. I... | Read more »
Let the world burn as you have some fest...
It is time to leave the world burning once again as you take a much-needed break from that whole “hero” lark and enjoy some celebrations in Genshin Impact. Version 5.4, Moonlight Amidst Dreams, will see you in Inazuma to attend the Mikawa Flower... | Read more »
Full Moon Over the Abyssal Sea lands on...
Aether Gazer has announced its latest major update, and it is one of the loveliest event names I have ever heard. Full Moon Over the Abyssal Sea is an amazing name, and it comes loaded with two side stories, a new S-grade Modifier, and some fancy... | Read more »
Open your own eatery for all the forest...
Very important question; when you read the title Zoo Restaurant, do you also immediately think of running a restaurant in which you cook Zoo animals as the course? I will just assume yes. Anyway, come June 23rd we will all be able to start up our... | Read more »
Crystal of Atlan opens registration for...
Nuverse was prominently featured in the last month for all the wrong reasons with the USA TikTok debacle, but now it is putting all that behind it and preparing for the Crystal of Atlan beta test. Taking place between February 18th and March 5th,... | Read more »

Price Scanner via MacPrices.net

AT&T is offering a 65% discount on the ne...
AT&T is offering the new iPhone 16e for up to 65% off their monthly finance fee with 36-months of service. No trade-in is required. Discount is applied via monthly bill credits over the 36 month... Read more
Use this code to get a free iPhone 13 at Visi...
For a limited time, use code SWEETDEAL to get a free 128GB iPhone 13 Visible, Verizon’s low-cost wireless cell service, Visible. Deal is valid when you purchase the Visible+ annual plan. Free... Read more
M4 Mac minis on sale for $50-$80 off MSRP at...
B&H Photo has M4 Mac minis in stock and on sale right now for $50 to $80 off Apple’s MSRP, each including free 1-2 day shipping to most US addresses: – M4 Mac mini (16GB/256GB): $549, $50 off... Read more
Buy an iPhone 16 at Boost Mobile and get one...
Boost Mobile, an MVNO using AT&T and T-Mobile’s networks, is offering one year of free Unlimited service with the purchase of any iPhone 16. Purchase the iPhone at standard MSRP, and then choose... Read more
Get an iPhone 15 for only $299 at Boost Mobil...
Boost Mobile, an MVNO using AT&T and T-Mobile’s networks, is offering the 128GB iPhone 15 for $299.99 including service with their Unlimited Premium plan (50GB of premium data, $60/month), or $20... Read more
Unreal Mobile is offering $100 off any new iP...
Unreal Mobile, an MVNO using AT&T and T-Mobile’s networks, is offering a $100 discount on any new iPhone with service. This includes new iPhone 16 models as well as iPhone 15, 14, 13, and SE... Read more
Apple drops prices on clearance iPhone 14 mod...
With today’s introduction of the new iPhone 16e, Apple has discontinued the iPhone 14, 14 Pro, and SE. In response, Apple has dropped prices on unlocked, Certified Refurbished, iPhone 14 models to a... Read more
B&H has 16-inch M4 Max MacBook Pros on sa...
B&H Photo is offering a $360-$410 discount on new 16-inch MacBook Pros with M4 Max CPUs right now. B&H offers free 1-2 day shipping to most US addresses: – 16″ M4 Max MacBook Pro (36GB/1TB/... Read more
Amazon is offering a $100 discount on the M4...
Amazon has the M4 Pro Mac mini discounted $100 off MSRP right now. Shipping is free. Their price is the lowest currently available for this popular mini: – Mac mini M4 Pro (24GB/512GB): $1299, $100... Read more
B&H continues to offer $150-$220 discount...
B&H Photo has 14-inch M4 MacBook Pros on sale for $150-$220 off MSRP. B&H offers free 1-2 day shipping to most US addresses: – 14″ M4 MacBook Pro (16GB/512GB): $1449, $150 off MSRP – 14″ M4... Read more

Jobs Board

All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.