The Not-So-Infinite Loop: Request-Response in WebObjects

By Sam Krishna and Patrick Taylor

In the beginning there was the Web...

When Tim Berners-Lee wrote Web.app (the first graphical web browser) using Nextstep tools, he couldn't have imagined the way the World Wide Web (1) would captivate an unsuspecting public. As revolutionary as it has become, Berners-Lee's creation wasn't a unique event. There were many predecessors to the Web which ranged from Vannevar Bush's ambitious, but ahead-of-the-technology, Memex to Ted Nelson's ambitious, but directionless, Xanadu. Unlike most of its predecessors though, the technology behind the Web was - and is - remarkably simple, especially when you consider how it is being applied today.

Most everyone knows at least a bit about how the Web works. With a browser that supports HTTP (hypertext transport protocol), a user connects to a remote server using an URL (Uniform Resource Locator) stored as either a bookmark or from a hypertext link on a document located at another remote location. The browser then proceeds to interpret the HTML tags in a text file on the remote server, formatting the text file based on the tagging. Hypertext links within index pages serve as springboards to other documents both within the current server or any number of servers around the world.

The number of those servers increased beyond anyone's reasonable expectations. By 1993, there were 50 web servers. Two years later there were 200 times as many. By 1998 there were well over 6 million and tens of thousands are added every day.

This was not the way the world was supposed to turn out. Back in the early 1990s, pundits were predicting that in the future information was going to be available to everyone on CD-ROMs with exciting multimedia interfaces: dictionaries, encyclopedias, games, cookbooks, repair manuals, baseball statistics; the knowledge of the world was going to be at your fingertips. As the speed of CD-ROMs leapt from 150 KBps to 300KBps and beyond, all manner of exotic data would be available ... for a reasonable price. It actually seems a bit funny looking back, but Bill Gates wasn't the only who didn't "get" the Internet until late 1995. Only the most visionary or lunatic individuals would have bet that the Web was the future much before then, though nowadays the whole world claims to have seen it coming as far back as 1991.

Of course, we know how things turned out. Tim Berners-Lee created the tool that provided the foundation for e-commerce, the rise of open source development as a mass movement and 11,000 point Dow Jones stock-markets. Would that all Nextstep apps have been so successful!

Good Enough, But Not By Itself

Berners-Lee wasn't out to revolutionize the world, but rather to create a useful tool that would let researchers at the European Center for Nuclear Research (CERN) share documents and data. Of course, it didn't take a nuclear physicist to see the advantages to hypertext linking and the simplicity of the Web's design over handling paper documents. But if all the Web had been good for was posting acronym-ladden documents and pictures of your cat, its growth might have stalled long ago.

Text and links gave way to a host of new datatypes. By adding new tags to a simple and flexible system, it was possible to extend the Web far beyond its early limitations: images, video, audio, animation and interactivity became realities as pioneering individuals pushed the envelope further and further and further.

Value versus Cost: the WebObjects Dilemma

I suppose this is as good a segue for what WebObjects costs as any. Unless you are fortunate enough to be an academic purchaser, a developer's licence for WebObjects/EOF will set you back US$1499. This isn't a licence to deploy only to develop, you will have to get a separate licence to deploy your application ranging from $5000 (for a 100 transactions per minute license) to $50,000 for an unlimited transaction, multiprocessor licence. Since every version of WebObjects includes the same software, all you need to do is enter a new licence key and the system will automatically accept a greater number of transactions (up to the limit allowed by hardware and the underlying operating system.

Academic developers can make out like bandits given Apple's incredibly generous academic bundle. For $99, academic (non-commercial) developers get a developers licence ($1499) and an unlimited transactions single-processor licence ($25,000). This may be the best reason yet to upgrade your university degree.

However, while the types of documents changed dramatically, the way those documents were served didn't change quite so much. Apache today is a much better web server than HTTPd was in the early 1990s, but it still serves static documents in basically the same way. You can create amazing web sites, however if you build them with straight HTML pages the sites can't scale very well. It's a hassle to rewrite HTML and the hassle grows geometrically with the number of pages and their complexity. And if it's difficult to rewrite your own HTML, imagine picking up after someone else!

If you want to go beyond static web sites (coherent collections of text-based HTML documents) you need to move to scripting or web application servers. If you don't mind hand-coding virtually all of the dynamic behaviour in your site, you may opt to use Perl or any number of scripting languages to dynamize your website. Don't fool yourself, remarkably sophisticated sites have been created with scripting languages alone, but not everyone is prepared to start from scratch. Or has the patience or time to do so.

The WebObjects Team

WebKit Bruce Ong Nico Popp

WebObjects 1.x - 3.1 Bruce Ong* Nico Popp* Charles d’Harcourt** François Jouaux (joined late in WO 1.x)

Architectural Consultants Marie Hulot Bertrand Serlet (acting manager) Craig Federighi

WebObjects 3.5-4.0 François Jouaux Charles Lloyd Eric Bailey Andrew Belk Craig Federighi (manager)***

* Left to found RealNames before WO 3.1 shipped ** Left after WO 2.0 *** Left after WO 4.0 for Ariba

And while WebObjects isn't the only web application server, it is the most mature. WebObjects backed by EOF is an industrial strength foundation for demanding web sites, which may sound like the PR material you'll read about most web application servers, but in this case it's real. If your site tops off at a half dozen pages, you definitely will not need WebObjects, but for anyone with complex web needs like those of web publications and e-commerce sites, you won't find software better than the WebObjects/EOF combo at any price.

The Web Now Available with the Power of Objects!

WebObjects started its life as a skunkworks project by Bruce Ong and Nico Popp, two quality assurance guys from the EOF and Openstep teams. Bring programmability and database access through an HTML interface was just one of those bright ideas that seems inevitable in retrospect, much like the Web itself. Under the codename "WebKit," management bought into the idea and the team created some basic demos including an application that rendered weather reports for windsurfers.

In its first two versions, WebObjects was built to work with the EOF 1.x stack and had an earlier, more limited form of state management but the outlines of the present architecture began to show. It wasn't until WebObjects 3.0 that the modern EOF 2.x-backed structure which still exists today (in a more refined form) came to be.

One of the goals of WebObjects was to provide as much of the power and flexibility of a desktop application over the Internet. Unlike desktop apps which respond to peripheral events, WebObjects apps respond to HTTP requests, which creates some limitations due to the stateless nature of the protocol (1).There are some basic generalizations one can make about the behaviour of a web transaction: a browser will initiate a request from a particular server, and that server will respond with either the required file or a message stating that the requested file is unavailable. By hiding behind an HTTP server and communicating through adaptors, a WebObjects-backed application doesn't look any different to a browser than a regular web server would, which is exactly what the designers intended. Other than a funny looking URL, nothing about the documents served from a WebObjects-based site should look different to a browser than a regular static document.

The play-by-play of how a WO application serves a document is quite simple:

1. The user's browser requests a page from an HTTP server
2. The server passes the request on to the WebObjects HTTP adaptor (which can use either CGI or, for performance reasons, a native plug-in architecture like NSAPI for Netscape servers)
3. The WO application calls takeValuesFromRequest() (Java) or takeValuesFromRequest:inContext (Objective-C)
4. The WO application sends a message to the class responsible for managing the page, if any exists
5. After all the appropriate classes have responded, the WO application collects the data to populate the required template, generates a new HTML page and then sends it to the WO Adaptor
6. The WO Adaptor passes the response to the HTTP server
7. The HTTP server returns the page back to the web browser

In most circumstances, you as a developer will only have to write code for steps 3 and 4, the rest of the behaviour is available as the default to the environment. WebObjects is not unique in providing some form of request-response cycle accessible to programmers even among other web application servers. What is different though is that WO requires virtually no programming in order to take advantage of the request-response cycle. In fact, in most circumstances it is unlikely that you'll need to directly intervene at all. This isn't the case for most other application servers which require far more hand-coding, often forcing the developer to write complex spaghetti code (3) for the most common situations.

When it comes to serving documents WebObjects takes a very agnostic view of the type of web server it prefers. The default adaptor it uses to communicate between itself and the HTTP server is the virtually omnipresent CGI. While common, however, CGI isn't well-suited to high-demand situations because it has to be reloaded for each and every request. So Apple provides different ways of hooking into HTTP servers that allow it. For servers with native plug-in APIs, specialized adaptors are provided to eliminate this potential bottleneck. An interesting result of this design is that the web application doesn't need to be hosted on the same machine (or even run on the same operating system or hardware platform) as the web server it only needs to know where the adaptor is, thereby allowing developers to create sophisticated load-balancing arrangements suited to an application's particular needs.

Strictly speaking, there are several more types of files associated with a web component. Compiled code specific to a component is stored outside of the page templates in .java (for Java, natch) or .m (for Objective-C) files. If you built the web component using Project Builder, a .api file is also created where you can place any API that should be public to other components.

Since a WebObjects-based site can potentially have multiple applications running at the same time, WebObjects attempts to simplify the process of ferrying information by placing directions within the URL it generates (this is the default, however much of this information can also be stored in cookies, as hidden fields in the HTML or some other customized session management approach). By parsing the URL, the adaptor can tell which instance of which application it should communicate with, the page requested, the requester's session ID and the elementID specified.

For example, parsing this URL tells us some interesting information:

The site I visited is the Apple Store, but in this case it was running an application called canadastore, which provides Canada-specific prices and information. For performance reasons, the developers set up the site to have multiple instances (the one I was on was number 155). FEc0D1WgREnfaBSGZ2 is the alphanumeric sessionID assigned to that particular transaction. And, 3.5.0.3.27.1 is the element ID assigned to the G4 I was looking at. These session and element IDs are ephemeral though for security reasons and will expire after a developer-determined period of time. When I looked at the site again after closing the window, the site returned the following URL:

As you can see it would be difficult to guess or even read the session information for a particular user making it incredibly difficult to hijack someone else's transaction. This information however is sufficient to overcome the empty, stateless divide between the WebObjects application and the user's browser.

Armed with this session information, the adaptor - whether CGI or one of the others - transmits HTTP requests it receives from the web server and passes them to the WebObjects application instance in a form it can understand. Alternately, on the return trip, the adaptor receives the HTML documents from the WebObjects application instance and passes it on to the web server for eventual delivery to the user's browser. This is the first part of the WebObjects request-response loop.

A Place for Everything, and Everything in its Place

There are several ways of expressing the development philosophy behind the tools Apple inherited from NeXT, but it all boils down to keeping different kinds of code separate. The interface code shouldn't mix with the business logic, which is kept distinct from the data. Part of the reason WebObjects apps are developed so quickly is because it is harder for you as a developer to write spaghetti code which other development environments actually seem to encourage.

Not to bash Bill Gates when he's down, but Microsoft is one of the worst offenders in this tendency to arbitrarily mishmash web application elements. Microsoft Active Server Pages places the HTML template code, the preferred HTTP response, the programming logic, and SQL commands all in a single ASP source file. For developers accustomed to the Model-View-Controller (MVC) approach of WebObjects/EOF and Cocoa, this is an excruciating mess. And if you imagine how much worse it becomes when you start involving database managers, graphic designers and HTML coders in your project, you may come to appreciate the anal-retentiveness of the MVC philosophy.

WebObjects creates these templates or web components using up to 4 types of files: a .html, a .wod, a .woo, a .wos. If you use compiled languages like Java or Objective-C exclusively instead of Apple's interpreted language,Webscript, you'll only have the first three types.

A .html file is the HTML template which collects different components (which can be made up of other HTML fragments) together and provide the shell into which WebObjects structures the data it gets most of the time from EOF. You would be able to recognize a WebObjects specific Form tag by looking for <WEBOBJECT NAME="Form1">...</WEBOBJECT>.

A .wod file is the WebObjects Declaration file, which assigns values to dynamic elements used in the HTML template. They can be either constants or bindings to variables, methods or entities. In the previous example WOForm would be assigned to Form1 to let the application know what type of WO tag it was dealing with. By separating the declarations from the interface, WebObjects makes the job of both the developer and the server easier. The developer is free to modify the interface without lots of declarations littering the code, while the server doesn't have to parse potentially huge files just to find declarations.

You will rarely need to manually edit a .woo file. It is used to setup DisplayGroups and stores information about which version of WebObjects was used to develop the app.

Clean and Simple

WebObjects gives the developers a number of vector points to modifying object behaviour without overwhelming them with choices. If you did want to change the standard request-response loop, there are three places where a developer can intercept and augment it: the Application class, the Session class and the individual WOComponent class for the given page. The appropriate place to intercept the request-response loop will be decided by whether you wish the behaviour to be global to your application, limited to a particular session, or a special case for a specific page. In any case, the Application class will be the first called when the loop is modified following by Session and Component respectively.

There are three methods which can directly interact with the request-response cycle:

Java											Objective-C

takeValuesFromRequest()		-takeValuesFromRequest:inContext:

invokeActionForRequest()		-invokeActionForRequest:inContext: 

appendToResponse()				-appendToResponse:inContext:

The method takeValuesFromRequest() is used to process the incoming WORequest object that the WOAdaptor creates immediately after processing the HTTP request the adaptor received from the HTTP server. This method is normally used to get form values and any other inputs that may have been passed through (like a radio button or checkbox selection). By overriding this method, you have considerable control over the request-response loop. [Examples???]

The method invokeActionForRequest() is used to actually call a component action that will return a page. If your application has mandatory fields in a form that a user might neglect to fill out, overriding this method will allow the application to redirect the user to the correct page after verifying that correct information doesn't exist yet. Rather than littering your HTML with conditional responses, the developer can create alternate documents which the application can serve depending on the circumstances.

The method appendToResponse() allows you to change the state of your component after it has been generated but before being relayed to the adaptor.You can perform last minute manipulations of your data before it is sent to the user. A potential use would be if you store some special state tht you don't want to leave in the session, so you could generate a cookie and append it to the response in the overridden method. As well, custom hand-coded HTML could be appended programmatically if you wished.

There are two methods that are important to note:

Java					Objective-C



awake				-awake

sleep				-sleep

The method awake is called every time a component is at the beginning of the request-response cycle. It's normally used by the developer to perform request-handling initialization. This method is different from either the constructor in Java or the -init method in Objective-C. The latter two are called once when the class is first instantiated the very first time the page is returned. The awake method is called whenever a page component begins a new request-response cycle and may also be sent several times during the same request-response cycle to the same component.

The sleep method is usually overridden in order to perform post-request-response processing. For example, you could begin manual garbage collection in Objective-C in the sleep method. As you can probably guess, this method is rarely used except in special cases where you must perform some sort of clean-up duties.

As stated before, most applications require little or no fiddling with the request-response loop. Once data has been pulled in from EOF, WebObjects constructs genuine HTML web pages that it passes on to adaptor. The adaptor relays it to the HTTP server which then transmits it to the browser. And the cycle can begin again.