TweetFollow Us on Twitter

Enabling CGI Scripts

Volume Number: 19 (2003)
Issue Number: 9
Column Tag: Programming

Untangling the Web

Enabling CGI Scripts

by Kevin Hemenway

Prepare your server for more powerful dynamic capabilities

Last month, we started down the road of adding dynamic content to our web server with the use of Server Side Includes (SSI). Often regarded as "simple" compared to languages like Perl and PHP, they can provide some quick turnaround to minor features like small single-item databases, changing content based on user whim or URL, and so forth. Regardless of whether you plan on using SSIs or not, we laid some important groundwork with our understanding of how modules work, Apache's "block" configuration format, and a soft introduction to GET and POST.

All relatively Lego. Let's break out the Technix.

A Quick 'How do ya do?' To CGI

SSI's are built into Apache through the use of a module--when they're enabled, the web server handles the request of a resource, processes the SSI statements within, and then sends the final output back to the user. Since Apache is a web server and not a programming language, the capabilities of the built-in SSIs are minimal--they're not intended to be a full-fledged coding environment, and they never will be.

This is where CGI comes in. Meaning "Common Gateway Interface", it defines a way for a web server to interact with external programs: Apache handles the request, sends some relevant information to the requested resource, and trusts the resource to handle the rest (including sending back the final content). If the resource doesn't respond properly (either due to bad programming, incorrect permissions, etc.), Apache generates an error.

What does this really mean? Ultimately, it allows you to use any shell programming language to "do stuff" before showing the results to your visitors. Whether this means displaying content from a database, resetting passwords, saving a comment to a weblog, etc., as long as your program finishes properly, Apache doesn't give a darn.

Most commonly, these "CGI scripts" are written in Perl, but you can use PHP, C, Bash, Ruby, Python, and anything else you may be interested in. Likewise, you can use them all in tandem--you're not locked into any one programming language at any one time. Be forewarned, however: you are locked into security concerns: anything possible in your programming language of choice is doable within your CGI script, including deletion of files, exhaustion of memory, buffer overflows, poor security, and impolite use of resources. Add in the fact that all of these errors in judgment become publicly run-able by anyone who accesses your server, and you've got a whole mess of potential trouble.

Enabling The Gateway to Your Scripts

Learning about CGI follows my oft-repeated method of "search the httpd.conf for the feature you want". So, open up /etc/httpd/httpd.conf in your favorite text editor and search for the word "CGI"--our first two matches look vaguely familiar:

LoadModule cgi_module         libexec/httpd/mod_cgi.so
AddModule mod_cgi.c

If you recall from our previous articles, Apache contains most of its features broken up into modules--the equivalent of plugins. To enable a module, two lines must exist uncommented (not preceded by a # character) in the httpd.conf file, an AddModule and a LoadModule. As previously with SSI, so too with CGI: our two lines already exist, and are already enabled. Let's move on to our next search result, which is also similar to what we saw last month:

<Directory "/Library/WebServer/Documents">
    # This may also be "None", "All", or any combination of 
    # "Indexes", "Includes", "FollowSymLinks", "ExecCGI",   
    # or "MultiViews".
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

Whereas last article the above result was triggered by the word Includes, it's ExecCGI this time around. Unfortunately, explaining ExecCGI without some prior knowledge will cause a little bit of confusion, so let's skip ahead to our next few matches before we go much further. I've truncated them for relevance:

# ScriptAlias: This controls which directories
# contain server scripts. ScriptAliases are essentially
# the same as Aliases, except that documents in the 
# realname directory are treated as applications and
# run by the server when requested rather than as
# documents sent to the client. 
#
ScriptAlias /cgi-bin/ "/Library/WebServer/CGI-Executables/"
# "/Library/WebServer/CGI-Executables" should be changed
# to whatever your ScriptAliased CGI directory exists,
# if you have that configured.
#
<Directory "/Library/WebServer/CGI-Executables">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>

The most common CGI configuration is all about segregation: "only files in this directory should be executed as programs--everything else, anywhere else, are just plain old files that should be processed normally." From the standpoint of an overbearing system administrator, this makes sense. With only one directory that can execute code unconditionally, and only one person (you) who can put programs there, it can bring a little sanity to your day-to-day security paranoia.

This is how Apache is configured by default, and the above httpd.conf snippet shows these settings. Our first directive, ScriptAlias, lets us pick and choose which directory we'd like CGI scripts to live in. Anything within that directory, CGI script or not, will be executed by the web server as if it were a program. If something goes wrong, Apache will return an "Internal Server Error" to the requesting user-agent (ie. your visitor's browser).

ScriptAlias consists of two space-separated parts: the fake name and the real name. The fake name, /cgi-bin/, defines what URL will be used to access the real name, /Library/WebServer/CGI-Executables/. In the currently configured settings, whenever we access a URL like http://127.0.0.1/cgi-bin/printenv, we'll really be accessing the matching file stored at /Library/WebServer/CGI-Executables/printenv.

The second part of our configuration is a block directive focusing on that specific location of our hard drive. It merely ensures that said directory has no special privileges besides that ScriptAlias: it can't override anything in the Apache configuration (AllowOverride None), and there are no special capabilities (Options None) associated with it.

At this point, if you only care about the directory segregation of CGI, you can skip over to our next subheading, "Let's See One of These CGI Thingies In Action!" If, on the other hand, you'd like to hear me get high-and-mighty over the "right" way of doing things, read on.

CGI EVERYWHERE: "A Brilliant Psychological Thriller!"

In the very first "Untangling the Web", I professed a fondness and desire to teach you the "right" way to create URLs (MacTech Magazine, June 2003, "Untangling The Web", "Familiarity That Breeds An Uh-Oh!"). A good URL will remain in existence until the dusk of time, never needing to be changed, never needing to be refactored. It's not a snap-decision: if you find yourself changing your URLs around, you designed your site poorly (from an engineering, not visual, standpoint).

One of the maxims, which I'll reiterate here, is that "Your URLs Should Not Reveal Your Technical Capabilities". This has absolutely nothing to do with the oft-maligned "security through obscurity"--the belief that hiding information makes you less susceptible to risk. Instead, removing the technicalities of your URL will allow your site to grow with your needs.

Consider the default configuration of CGI: anytime you want to add some dynamic functionality to your site (with Perl, Python, etc.), you've got to stick your script in a directory and point to it with a URL like http://127.0.0.1/cgi-bin/add_user.pl (where .pl denotes a Perl script) or http://127.0.0.1/cgi-bin/del_user.py (.py denoting Python). What happens, though, when you no longer need CGI scripts, but have moved to an embedded language like PHP? Suddenly, all of your pages that point to that ugly /cgi-bin/ URL are broken, useless, and inaccurate. You find yourself refactoring pages to point to new locations.

The solution? Remove /cgi-bin/ and the file extension from the URL. By clearing this needless technical cruft, you get stronger addresses with less need for change. http://127.0.0.1/admin/add_user and http://127.0.0.1/admin/del_user contain no indication of the technology behind them, merely their purpose in the grande scheme of things.

We're already halfway through removing /cgi-bin/ from our URLs (file extensions I keep putting off, but I'll get to 'em eventually, I promise!). Our very first match for our "CGI" search was for an ExecCGI addition to the Options line of our root web directory (/Library/WebServer/Documents/). ExecCGI works similarly to Includes, which helped get our SSIs running last month. By adding it to the Options line of any Directory we, like SSIs, are telling Apache to allow CGI scripts at that location:

Options Indexes FollowSymLinks MultiViews ExecCGI

To make the comparison even more apt, there's one other thing we need to do before we can use CGIs wherever we need them: we need to add a handler that says "Apache! Listen up! Whenever we request a file with this extension, we want you to treat it as an executable program!" This handler was also needed last month--we told Apache that anytime it ran across a file with an shtml extension, it should process it for SSIs statements.

Thankfully, there's not much brainwork involved here, as our final relevant search match for "CGI" brings us to the below. Simply uncomment the AddHandler (by removing the #), restart Apache (sudo apachectl restart in your Terminal) and you're ready to go:

# If you want to use server side includes, or CGI outside
# ScriptAliased directories, uncomment the following lines.
#
# To use CGI scripts:
#
#AddHandler cgi-script .cgi

Two caveats that may be running through head: "what about the cgi extension?" and "what about that security brouhaha you cautioned us about?" Concerning the file extension, yes, we're swapping /cgi-bin/ for cgi, but this will never be an issue once we remove the need for them in our URLs. As for security, there's not much I can say besides "be careful"--you'll no longer have the niceties of a single and central directory for your code execution, so double and triple check the scripts you'll be programming or using. I'll talk a little more about script security in next month's column.

Let's See One of These CGI Thingies In Action!

Browse to your /Library/WebServer/CGI-Executables/ directory, the default location that Apache has been configured for CGI scripts. You should see two files, printenv and test-cgi, there already. These are some default Apache CGI scripts that will help you quickly test if things are a-ok. Since CGI was already pre-configured in the httpd.conf, we should be able to just load one of these files in the browser and see some fireworks, right? Load http://127.0.0.1/cgi-bin/test-cgi, and we'll receive the response in Figure 1.


Figure 1: An error after loading our test-cgi, but why?

If CGI scripts were already configured, why this rather rude error message? Let's check Apache's error_log, which will always contain some sort of light bulb enlightening response. After running tail /var/log/httpd/error_log on the command line, we'll see something similar to the below as the last line of output:

[Sun Aug 17 10:41:16 2003] [error] [client 127.0.0.1] file 
permissions deny server execution: /Library/WebServer/CGI-
Executables/test-cgi

This is one of the more common errors you'll experience with CGI scripts: the test-cgi file doesn't have the proper credentials to be considered an executable program. I won't get into the vagaries of file ownership or permissions, but follow through the bolded commands below to give both of the default CGI files "execute" access for all users:

> cd /Library/WebServer/CGI-Executables/
> ls -l
total 24
-rw-r--r--  1 root  admin  5398 Jul 27  2002 printenv
-rw-r--r--  1 root  admin   757 Jul 27  2002 test-cgi
> sudo chmod 755 *
Password: **********
> ls -l
total 24
-rwxr-xr-x  1 root  admin  5398 Jul 27  2002 printenv
-rwxr-xr-x  1 root  admin   757 Jul 27  2002 test-cgi

Now, let's try our URL again. Figure 2 shows our new output:


Figure 2: The correct output of our test-cgi script.

Figure 3 shows the output from the other script, http://127.0.0.1/cgi-bin/printenv:


Figure 3: Similar output from the default printenv script.

If you followed the instructions under the "CGI EVERYWHERE!" subheading, you can also move either of the printenv or test-cgi scripts out of their current location and stick them in /Library/WebServer/Documents/ with a cgi file extension. Once you do so, you should then be able to run them under the non-/cgi-bin/ URL and receive the same output (if not, what's your first course of action? Check the error_log!).

Assuming you get similar output as Figures 2 and 3, what does all this gibberish mean? What exactly did we just do? Both scripts show a number of environment variables, which is just a fancy term for data that exists "invisibly" and floats around waiting to be used or read. Some of the variables were set by the browser (like HTTP_USER_AGENT), and some have been set by the server (like SERVER_PROTOCOL). When I first introduced CGI, I mentioned that Apache "sends some relevant information to the requested resource". This "relevant information" is stored in environment variables.

Environment variables can exist outside of your web server too. For instance, if you run printenv in your Terminal, you'll see a listing of variables similar to Figure 4:


Figure 4: Environment variables in your Terminal.

Believe it or not, we've dealt with environment variables already in this series. In our article on Server Side Includes, we checked the $QUERY_STRING variable to see which quote should be displayed at the user's request. If you check back to Figure 3, you'll see an environment variable named QUERY_STRING, which is currently empty. Try adding ?q02 to the end of our URL (http://127.0.0.1/cgi-bin/printenv?q02), and see how that changes the output (Figure 5):


Figure 5: Our QUERY_STRING now has a value.

With the above two scripts working correctly, our CGI capabilities are ready for use.

Homework Malignments

In our next column, we'll chat further about CGI: how to begin creating our own, the most common errors during development and deployment, security issues we should be aware of, and how to find the "good ones" from the immense ocean of crap. If we have time, we'll show you how to install a few that have stood the test of time. For now, students may contact the teacher at morbus@disobey.com.

  • Take a look inside printenv and test-cgi in a text editor.

  • View the printenv script from various browsers and machines. Determine how the output changes, and see if you can figure out what each setting actually means.

  • Forgive me for not including anything amusing in this article.


Kevin Hemenway, coauthor of Mac OS X Hacks, is better known as Morbus Iff, the creator of disobey.com, which bills itself as "content for the discontented." Publisher and developer of more home cooking than you could ever imagine (like the popular open-sourced aggregator AmphetaDesk, the best-kept gaming secret Gamegrene.com, articles for Apple's Internet Developer and the O'Reilly Network, etc.), he's been spending the last two months listening to every song in his iTunes library twice. Contact him at morbus@disobey.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

TotalFinder 1.12.2 - Adds tabs, hotkeys,...
TotalFinder is a universally acclaimed navigational companion for your Mac. Enhance your Mac's Finder with features so smart and convenient, you won't believe you ever lived without them. Features... Read more
Duet 2.3.0.3 - Use your iPad as an exter...
Duet is the first app that allows you to use your iDevice as an extra display for your Mac using the Lightning or 30-pin cable. Note: This app requires a $9.99 iOS companion app. Version 2.3.0.3:... Read more
FileMaker Pro Advanced 18.0.3 - Powerful...
FileMaker Pro Advanced is the tool you use to create a custom app. You also use FileMaker Pro Advanced to access your app on a computer. Start by importing data from a spreadsheet or using a built-in... Read more
OsiriX Lite 10.0.6 - 3D medical image pr...
OsiriX Lite is an image processing software dedicated to DICOM images (".dcm" / ".DCM" extension) produced by medical equipment (MRI, CT, PET, PET-CT, ...) and confocal microscopy (LSM and BioRAD-PIC... Read more
Ableton Live 10.1.5 - Record music using...
Ableton Live lets you create and record music on your Mac. Use digital instruments, pre-recorded sounds, and sampled loops to arrange, produce, and perform your music like never before. Ableton Live... Read more
Burn 2.7.8 - Easily burn data, audio, vi...
Burn... There are a lot of ways to approach burning discs. Burn keeps it simple, but still offers a lot of advanced options. Create data discs with advanced data settings like, file permissions, the... Read more
Malwarebytes 4.0.30.3073 - Adware remova...
Malwarebytes (was AdwareMedic) helps you get your Mac experience back. Malwarebytes scans for and removes code that degrades system performance or attacks your system. Making your Mac once again your... Read more
Acorn 6.5.3 - Bitmap image editor.
Acorn is a new image editor built with one goal in mind - simplicity. Fast, easy, and fluid, Acorn provides the options you'll need without any overhead. Acorn feels right, and won't drain your bank... Read more
Fantastical 2.5.13 - Create calendar eve...
Fantastical is the Mac calendar you'll actually enjoy using. Creating an event with Fantastical is quick, easy, and fun: Open Fantastical with a single click or keystroke Type in your event... Read more
A Better Finder Rename 11.05 - File, pho...
A Better Finder Rename is the most complete renaming solution available on the market today. That's why, since 1996, tens of thousands of hobbyists, professionals and businesses depend on A Better... Read more

Latest Forum Discussions

See All

Eternal Warfare is a new idle clicker fo...
Idle games are a popular genre on mobile, they might not be to everyone's taste but they're made with such regularity and receive a lot of downloads, so it's hard to argue it's not big business. Eternal Warfare is set to join the sea of idle games... | Read more »
New heroes and balance updates set to ar...
It feels like Hearthstone: Battlegrounds only launched yesterday, and already the auto batter addition to Blizzard's megahit card game is set to receive new heroes and balance updates. [Read more] | Read more »
Pre-register for Hello Kitty AR: Kawaii...
Hello Kitty — the cute cat that launched a multi-billion-pound franchise — has been brought to life… sort of. Sanrio has teamed up with the Bublar Group to create a new mobile game that uses AR tech to turn the real world into Hello Kitty’s... | Read more »
Gorgeous and tranquil puzzler Spring Fal...
One-man indie studio SPARSE//GameDev has now launched its tranquil puzzler, Spring Falls. It's described as "a peaceful puzzle game about water, erosion, and watching things grow". [Read more] | Read more »
Black Desert Mobile gets an official rel...
Pearl Abyss has just announced that its highly-anticipated MMO, Black Desert Mobile, will launch globally for iOS and Android on December 11th. [Read more] | Read more »
Another Eden receives new a episode, cha...
Another Eden, WFS' popular RPG, has received another update that brings new story content to the game alongside a few new heroes to discover. [Read more] | Read more »
Overdox guide - Tips and tricks for begi...
Overdox is a clever battle royale that changes things up by adding MOBA mechanics and melee combat to the mix. This new hybrid game can be quite a bit to take in at first, so we’ve put together a list of tips to help you get a leg up on the... | Read more »
Roterra Extreme - Great Escape is a pers...
Roterra Extreme – Great Escape has been described by developers Dig-It Games as a mini-sequel to their acclaimed title Roterra: Flip the Fairytale. It continues that game's tradition of messing with which way is up, tasking you with solving... | Read more »
Hearthstone: Battlegrounds open beta lau...
Remember earlier this year when auto battlers were the latest hotness? We had Auto Chess, DOTA Underlords, Chess Rush, and more all gunning for our attention. They all had their own reasons to play, but, at least from where I'm standing, most... | Read more »
The House of Da Vinci 2 gets a new gamep...
The House of Da Vinci launched all the way back in 2017. Now, developer Blue Brain Games is gearing up to deliver a second dose of The Room-inspired puzzling. Some fresh details have now emerged, alongside the game's first official trailer. [Read... | Read more »

Price Scanner via MacPrices.net

B&H offers $100 discounts on 4-Core and 6...
B&H Photo has new 4-Core and 6-Core Mac minis in stock and on sale today for $100 off Apple’s MSRP. Prices start at $699. Overnight shipping is free to many addresses in the US: – 3.6GHz Quad-... Read more
Save $200 today on a 2019 13″ MacBook Air wit...
Apple has a full line of Certified Refurbished 2019 13″ MacBook Airs available starting at only $929 and up to $200 off the cost of new Airs. Each MacBook features a new outer case, comes with a... Read more
New Verizon Pre-Black Friday 2019 deal: Buy o...
Buy one new Apple iPhone 11 model or 2018 iPhone XS model at Verizon and get a second one for free. One new line of service required. Offer is valid from November 21, 2019 to November 27, 2019. Here... Read more
AirPods with Wireless Charging Case on sale t...
Abt Electronics has 2019 AirPods with the Wireless Charging Case on sale today for $163 shipped. Their price is $36 off Apple’s MSRP, and it’s currently the cheapest price for these AirPods from any... Read more
Apple continues to offer 2017 13″ Dual-Core n...
Apple has Certified Refurbished 2017 13″ 2.3GHz Dual-Core non-Touch Bar MacBook Pros still available starting at $1019. An standard Apple one-year warranty is included with each model, outer cases... Read more
Save up to $120 on the new 16″ MacBook Pro at...
Apple’s resellers are starting to receive stock of new 16″ MacBook Pros, and the first set of sales & deals are now available: (1) Amazon 16″ MacBook Pros start on sale for $100-$116 off Apple’s... Read more
Apple Watch Series 3 models on sale at Amazon...
Amazon has Apple Watch Series 3 GPS models on sale for $30 off MSRP, starting at only $169. There prices are the lowest we’ve ever seen for these models from any Apple reseller. Choose Amazon as the... Read more
The ‘Mac Potpourri’ Mailbag: Edition #1- Info...
COMMENTARY: 11.20.19- Welcome to the inaugural edition of the “Mac Potpourri” Mailbag where we take a look at correspondence received from readers of this column from all over the world who write in... Read more
13″ 2.4GHz MacBook Pros available for up to $...
Apple has a full line of Certified Refurbished 2019 13″ 2.4GHz 4-Core Touch Bar MacBook Pros available starting at $1529 and up to $300 off MSRP. Apple’s one-year warranty is included, shipping is... Read more
New at T-Mobile: Switch to T-Mobile, and get...
T-Mobile is offering a free 64GB iPhone 8 for new customers who switch to T-Mobile and open a new line of service. Eligible trade-in required, and discount applied over a 24 month period. The fine... Read more

Jobs Board

Best Buy *Apple* Computing Master - Best Bu...
**747303BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Store NUmber or Department:** 001413-Cypress-Store **Job Description:** **What Read more
*Apple* Mobility Pro - Best Buy (United Stat...
**743221BR** **Job Title:** Apple Mobility Pro **Job Category:** Store Associates **Store NUmber or Department:** 000230-Greenwood-Store **Job Description:** At Best Read more
*Apple* Mobility Pro - Best Buy (United Stat...
**747338BR** **Job Title:** Apple Mobility Pro **Job Category:** Store Associates **Store NUmber or Department:** 000254-Superstition Springs-Store **Job Read more
Best Buy *Apple* Computing Master - Best Bu...
**745516BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Store Associates **Store NUmber or Department:** 001101-Manhattan-Store **Job Read more
Best Buy *Apple* Computing Master - Best Bu...
**746655BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Store NUmber or Department:** 002518-Atlantic Center-Store **Job Description:** Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.