Volume Number: 16 (2000)
Issue Number: 10
Column Tag: Managing a WebObjects Project

WebObjects Development Lifecycle

by David K. Every

What does WebObjects do?

Article Scope

There are many parts to making a successful web-solution. The success or failure will be based on how good you (or your company) is at being an engineering company, engineering team or at least engineering project. If you aren't an entire engineering company then you are going to have to emulate all the facets of being one. If you don't have the size and experience, and are approaching the problem on your own or with a small team, then you need to try to all wear many hats in order to make sure all the parts of the project run smoothly. Each part is important and will help you avoid pitfalls - so don't discount them.

It is beyond the scope of this article to explain all the concepts necessary to manage a successful team, how to do proper analysis, scoping, design, project management, as well as explaining how to be an interface designer, DBA and data architect. But it can sum up some of my experiences (many hard learned), and hopefully give the reader an introduction or review of the high level concepts on what to do, what not to do, and what are the warning signs to look out for.

This article is also tailored towards WebObjects project management - assuming an "average" project. Your project might not be average - but you can think about how you can borrow ideas and concepts, and how this information might help you..

Balance

The key to creating a successful project is balance. Always remember the balances and tradeoffs. You have to balance your goals, with the customers desires. You have to balance marketing with engineering and with financing. You have to balance features (power) against simplicity (learnability) - and form (style) against function (usability). You need to know your customers (marketing and feature), but you need to know what you can develop (engineering) - all balanced with what you can afford in time and money. There are many seemingly conflicting goals, and all of them have to weighed against the others and balanced. Everyone is not going to get exactly what they want - but if each gets to keep the important things, and they get it in balance with the others, then the product will probably be better than if there is no balances at all.

Balancing the needs (and communicating them) is done in order to get everyone on the same page, and shooting for the same goal. Even if meetings seem useless or time wasting, they are valuable if you can build some consensus or let people feel like their views have been aired (and really listened to). Even if they don't get what they want, you are more likely to get buy-in if they feel they have a voice and win some of the time. Even spending time exploring paths you aren't going to follow can be very important, if they can at least explain why the company isn't going to do something, or why things are being done the way they are. This makes your decisions more defensible and better understood. So meetings can be a tool to create consensus and understanding or they can decay into adversarial places where there is little real communication - always strive for the former.

Conflict is going to be natural in the process - people are each advocating different views or different parts of a solution which are critical to them. Accept that conflict, don't take any of it personally, and don't hold grudges and try not to allow others to. You need to express a view strongly, and get over the fact that others won't agree and you aren't going to get everything that you want - if you can't do that, then you certainly shouldn't be involved in engineering. Remember that you all have the same goals - to build the best product possible, in the least amount of time (with the lowest costs). Engineering is about tradeoffs and balancing all the differing desires - but you are all shooting for the same prize. Keep reminding people of that, and keep reminding them that things will evolve and not getting something now, may only mean for a while. Good ideas will keep coming back, and eventually win - and often things lose because it is just not yet the right time for that goal, but not that their goal itself is bad.

Unfortunately there is no magic to knowing where the lines should be drawn, when a project is unbalanced, and what to do about every situation. Nor is it easy to know where to cut off communications on certain paths, or all the nuances of managing people and delicate professional egos. That is all stuff that you just have to learn by experience, and there are books more targeted towards those subjects. However, if you at least remember to watch for balances, make sure that people are being heard and that your team is building consensus (and not hostility), people are learning why and where compromises are being made, and you are keeping communication open and constructive (with the eye on the prize), then at least you have a head-start on creating a successful project.

There are many books on the theory of analysis, design, and development. Many explain how to decompose problems from different points of view, and different methodologies for OOA (Object-Oriented Analysis) or OOD (Object-Oriented Design) - and many more on the older procedural analysis and design processes. Some explain techniques for drawing how things relate, nomenclatures, methodologies, and mental exercise to break problems down, and so on. This article is not one of those books. I've read many of those books, and I'm not knocking them - they have value and I recommend reading many of them, they have many nice insights that can help. However, most companies just don't have a staff of people all trained in the same methodology, at all levels of the company, in order to do a good job with them. Without a deep understanding of the processes (at the management as well as at the engineering level ) then the processes can quickly become less than efficient - and the results will probably not meet expectations.

The point of what analysis and design is really about should be as much about getting people on the same page, getting them communicating well, managing expectations and getting some consensus and agreement on what will be done. It is setting targets that people understand and can shoot for, and measure progress against those targets so you can get better over time. The processes, documentation and drawings, and exercises in many analysis books and classes are not the ends - they are just ways to try to achieve the ends (communications, consensus, repeatability in forecasting, setting goals and managing expectations).

Many of the larger, complex processes may be necessary and valuable on larger projects (say 15 engineers or more) with multiple teams, huge development cycles (over a year), and huge amounts of investment. However, successes in those processes are far more rare than the failures - especially if people aren't willing to be adaptive, or willing to accept the facts on scheduling or scope that is often staring them in the face. I've found that those processes are usually overkill for creating most web-applications.

WebObjects is a powerful rapid application development tool that makes programmers much more efficient than many other tools. This magnification of programmers efficiency means that smaller teams can do more - and thus more projects are smaller with WebObjects than they would be without it. So the project management processes (including analysis and design) can be a little bit lighter as well.

If you have a large team, of highly trained people, or a well established process that is working for you, then I'm certainly not suggesting that you throw it all away and try to mimic the suggestions of this article. Stick with what works. But there are many who are unsure about how to approach WebObjects development, or looking for alternative ways of thinking about that problem. This article is about techniques that I've learned and used in startups and larger companies to help establish a sensible process (or streamline one) for doing WebObjects development and project management. These are ideas for how to manage a WebObjects project (smaller, rapid development projects), while still maintaining a useful amount of documentation and process in order get things done right, and to still allow engineering enough critical communication to work well with other groups such as Quality Assurance, Documentation, Marketing, Management and so on.

Go Visual

If you verbally describe something to a group of people they will each imagine something different - they may think that they are all talking about the same thing, but in their minds they are picturing something that might be quite different from the rest. When one person delivers on what they were saying, the others often find that it didn't meet with their expectations - and nothing seems to get people more frustrated then having something fail to meet their expectations (ask any married man). The further away you are from their expectations, the worse things are likely to be - so one of the keys to successful project management, and a happy life, is good communications and managing expectations.

If you write a document that describes how something works, people are more likely to be on the same page than just speaking the same thing. Speech is more imaginative than writing, and words on a page can be less ambiguous. Also people can digest written documents at their own speed, and review until they get it. So a well written set of design documents can really help facilitate communication.

Unfortunately, people are still victims of their ability to write, read and digest these documents - and those are skills that take time to master. Most design documents are not very good and tend to take up lots of time to create, with little results, and tend not to survive confrontation with implementation - so by the time the product ships they are woefully out of date. Many people aren't willing to read long design documents - and just ignore them. So while design documents are usually better than just verbal concepts, written words usually aren't good enough to get people to understand what others mean.

Most people tend to be visual and understand what they see, and they don't have to conceptualize the intangible based on your description. Thus the key to successful documentation (or many discussion) is pictures and drawings. This is why many restaurant menus have pictures - don't describe it, just show it. Don't be afraid of a white board, a sketchpad, or a drawing program. It is cliche, but sometimes a picture really is worth a thousand words - and people are more likely to use the picture than they are to read the thousand words. A few simple drawing, with notes and points can replace dozens of pages of wordy documentation. Because of this, I find to be the best form of design document, and tool for project management is the prototype!

Prototyping is the quickest way to get people to really envision the goals. And the quickest prototypes I've ever used are just a series drawings, with a few notes to describe what each thing does, and where items are labeled to go to other drawings. At the first phase, I used to do this with a bunch of printed pages, and page numbers for each link, button or image, and just through the stack to show people where each was going. Later, I automate the navigation with some prototyping tool (like HyperCard or VisualBasic). With HTML and websites, it is even easier to just link a bunch of static pages together and use that as a prototype that people can see and feel and sign-off on.

With all this in mind, here is my simple process for analysis, design, documentation and project management.

1. Analysis starts with information collection. Collect all the features and ideas that people want, and build a list (so many ideas won't be lost). And this is just the features and ideas people want for the first public release - keep a separate list for the far future stuff (that list will be revisited after you've shipped 1.0).
2. Prioritize the 1.0 list, and divide it into three or four groups. Those groups will be iterations of releases. (I'm a big fan of iterative development - or at least a hater of waterfall development, since I've never seen the latter actually meet expectations, and it always causes undue stress and frustration). The first few iterations are your targets for the first release - the last few are your visions for the future or "nice to have's".
3. Most of the effort should be focused on things that will make the cut. You can, and should, capture a few good ideas that can be in the other passes - but focus on getting people through the first iteration of the cycle. Remember to layer functionality - get the basics working, then add the "extras" later on. Let people make their mistakes early in the process when they are less costly - let them learn what to do and what not to do before you are under the high-pressure end-game. The key to the early passes is to learn.
4. Now go on to the next step and use the design process to mature your analysis - you don't have to do all the analysis in the first stage of the first iteration of the process, let the next step mature the previous one. The goals are to get most done and revisit over time - not get everything done at once. Feature creep, design creep, and code-creep happens. Don't fight it - just plan for it. Like a fine wine, let things mature on their own time.

The steps sound simple - but it can take quite some time just to get through that. In order to keep on track, try to keep the analysis from getting into specifics and details - it is easier to just manage the higher level feature list, without letting people war over the details of implementation. Don't try to do it all in the first pass. Make sure people realize that there are other iterations in the process - and that things will get in and that later planning will get into details.

No analysis is perfect. In fact, I've rarely ever seen analysis that was really close to the final product - and many of those that were, turned out to be exactly what was asked for, just not what people wanted. No one is omniscient, and since you can't anticipate everything up front, things are going to have to change. So you have to be adaptive, don't try to capture everything - just try to capture what you have at each step. Keep coming back, and keep things up to date - but a series of small meetings and capturing what you have gives people a sense of accomplishment. Then come back and keep making those baby steps. If you try to do it all in one series of meetings, everyone will end up frustrated or bored, and you'll miss major chunks of functionality and lots of good ideas. If you keep revisiting over time, and capturing a few things on each pass, people get renewed and refreshed to add more each time.

Tip: It is better if you don't go into meetings empty handed. It will dramatically reduce the time to develop if someone is in charge of being the architect, or chief interface designer, leader, and they come into the meetings with a rough draft already sketched out. That person can capture half of the information via one on one meetings with the others before the bigger collective meetings - this is a big head start. When a meeting starts, it should be a continuation of where people left off - and just be about capturing any extra stuff that what was left out. Collaboration is useful but time consuming - so if you can get all the common stuff out of the way, then you are only left with the important part of collaboration (which is the areas that need to be discussed, and not all the stuff where people are in fierce agreement). This head start will get meetings completed in a fraction the time - and a good moderator (who keeps things on track, and schedules follow up issues for other meetings) will help get meetings over in a fraction the time as well. This makes meetings more useful, productive and satisfying.

Another important meeting tip is the duration. 1 hour meetings are about optimum (unless you can do them in less time). After a couple hours, people completely glaze over and start nodding to anything just to end the meetings. This is the EGADs factor - "Everyone just doesn't Give-A-Damn" when a meeting is running too long - they are just thinking of escape. The only thing worse than this, is the personalities that may become so pissed off (they're tired and frustrated), that the meetings turn hostile, and the meetings turn into a slug-fest. So it helps in analysis and design meetings (or all meetings for that matter), to break them into many smaller meetings (one or two per day), than to try to rush it and cram it all in at once. Let the process happen. If people start getting tired and cranky or apathetic, it is time to break it up and come back on a day when you'll be more productive. And sometimes if people are tired of one subject, it is better to just skip it, and address the subjects you can agree upon, and try to come back to "sensitive issues" later (another day or week).

High Level Design

Once you have a working set of high level requirements (which is what the analysis should have given you'll), use the design process to mature the requirements and create the lower level requirement. (High level design and low-level requirements are almost the same thing).

For a website, the high level design starts at the navigation, and then putting what function is at each point in the navigation. That is it.

Build a hierarchy (outline) of the site map, and how a user will navigate.

On larger sites there may want to break the site into a few (or many) smaller sites. Each small site will be around a branch of functionality, or a role.

Add in a list of functions for what each stop on the hierarchy should have on it (what features and functions are available).

Other issues and ideas will be brought up that will need to be added to the requirements, or at least discussed as requirements. Don't try to solve them all at this level - maybe adding one or two of the obvious ones won't slow you down, but collect a list of any of the more ambiguous ones, or ones that are going to cause contention to bring back to the analysis review meeting. This will keep the design meeting moving - solve the problems you can solve, and skip the ones you can't. And this will also allow the design to mature the analysis.

At the end of this, you should have a site map - which is a nice visual way for you to see how people will navigate, and where they will get to various features and functions. This visual representation of where things go will help things mature much faster than it would have as just a spec. And you will be able to see that all the functions are captured and available somewhere.

Tip: I tend to use the web to design the web - since I believe the purpose of the web is communications, this is "eating your own dog food". I make an intranet site for development that contains all the notes/documentation from each step of the process. That site would contain (at this point) a simple high level requirements document, and a static site (navigation) map with each page being a list of functions that page will do (and maybe what it needs to contain). You can do this on paper - just the website is more interactive and publicly available, is easier to update, and saves trees. The designers and principals can now see what is where, and truly feel the hierarchy and the beginnings of a product. Everyone will start understanding the scope of the product and feel progress.

Low Level Design

Get an interface designer and graphic artist to create the first pass look. I call this the style-guide - what is the style and look of the site. This is just a simple picture for what each types of page will look like. Get a first pass of this done sooner because it gives people a lot of satisfaction to see some of the goals they are shooting for. Avoid needless battles over details, remember to remind people that this is just a very rough first cut (to get started) - and the end result may not look anything like this.

Decompose the common elements - both visually and functionally. I often do this with a User Interface Guidelines document that has a list of what rules that I'm going to try follow. Know that this too will mature over time - so it starts small and loose, and will fill in as you go. Create a list of common features and functions that are repeated on many pages - and try to put these on all (or almost all) pages, and in standard places. This will allow reuse of code, and create an interface metaphor (set of common behaviors that users will learn).

Now revisit each stop on the site-map - and instead of having just the feature list on each page, create a simple sketch of where things will be. Since you have set of UI rules for how things will behave, and a first pass at what things will look like, these sketches only really need to capture placement since you already have more details about look captured in the graphical style document, and the details of the feel is captured in the UI document. I usually use rectangles with what is going to go where, though some people want more detail and like to actually put in place detailed picture from the UI guidelines and Graphical look to make a detailed mockup.

Again, this phase of the process will mature the first two (analysis and high level design). The low level design will probably require changes and maturing of the high level design, and to a lesser extent modify the requirements. So capture any issues that are brought up, and quickly get past the ones you can't answer. Do what you can. Let things happen in their own time.

At the end of this, you not only have a site map, but you should be able to walk the pages with not only a list of what is going to be on each page, but a little sketch of where things might be placed. You also have a rule book for how you'd like things to behave. (Of course guidelines are not rules - just goals. They will have to change over time as well, and there will probably be exceptions - but the goal is to follow them as much as possible, and any deviations must have good reasons). Lastly you have a feel for what things will look like with the style guide.

Tip: Again, I tend to use the web to design the web. I tend to put the interface guidelines that are created, and the general look of the site on the developers intranet site. Since you already have a map how the site will navigate, with what features it will contain (place holders for each page), now you can just add what it will look like (roughly) on each of those pages (in place). I put in the sketches for each page, on each page in the hierarchy (with some direct navigation links). This allows a simple working mockup, where people can navigate page to page, and see what each page might look like (very roughly) and what will be there. At the bottom of each page is a list of things this page should do, or at least a link to a page that has that information. And you need a section of the site to map what the common elements (that will be reused on many pages) will look like and need to do.

Data Model and Back-End

Now that you have collected the basic features and requirements for the interface, the database requirements should all be laid out for you.

• Collect all the data elements that will be needed from each of the screens.
• Pool the data in logical groupings - what belongs with what. How do they relate to other items?
• Normalize the data so that common elements are in their own tables (and don't have to be entered more than once), and group data in a logical manner (often as close to mapping to the screens as is easily possible) and you should have a pretty good back end.

There is a lot to doing a good data map that is beyond the scope of this article - just like there is a lot to doing good graphic arts, or good UI design that is also beyond the scope of this article. But at least these are the steps you can can take.

This step in the process will help you find elements that you forgot for other pages (low level design), and can easily ripple up to the high level design and requirements. Go with it. After this stage you should have a system design that people can really shoot for. You have the top, and bottom. You have the basics of a look and a feel. You also have the behaviors of what each screen is going to have to do. You are now ready to go.

Tip: I tend to use the web to design the web - since I believe the purpose of the web is communications, this is "eating your own dog food". I make an intranet site for development that contains all the notes/documentation from each step of the process. That site would contain (at this point) a simple high level requirements document, and a static site (navigation) map with each page being a list of functions that page will do (and maybe what it needs to contain). You can do this on paper - just the website is more interactive and publicly available, is easier to update, and saves trees. The designers and principals can now see what is where, and truly feel the hierarchy and the beginnings of a product. Everyone will start understanding the scope of the product and feel progress.

Middle Layers (Business Logic)

The last thing you must do is start figuring out how to hook the back-end to the front end, and the design is done. On many small or mid-sized projects, it will go pretty quickly and there will be little need for a lot in-between (all the mid layers). On larger project, or complex project this can be quite an effort - but all this is traditional, and WebObjects doesn't change how these processes are done.

Often the list of features that are on the site map are the business rules. The what does this page have to do (from a business sense). Once you start trying to implement them, you find out that those high level rules may take a lot of back-end code. Many of these will be your middle layer. If you embed them in either the front or back end (interface or database respectively), then you will usually make it harder to evolve and grow those over time. So often these rules and functions go in their own middle layer.

The other thing that the "middle layer" is for is a place to put commonality and things that you can reuse. Just about anything that is going to have to be used on more than one page, or on more than one entity, is a good candidate for the middle layer. This improves code reuse - which means writing less code, supporting less code, and of course few bugs (since repeated behaviors are implemented in only one place - so one fix, fixes all cases where that bug can manifest).

If you just keep a watch out for things that can be reused, or things that don't really belong in the interface, nor in the entities themselves - then the middle layer will mature itself, and all you should do is just keep a running list of what is in there, and where it is located.

Project Management

To do project management, you need to manage time, resources and scope. You can't do that effectively until you understand scope. Every step in the process was just leading to this point. Now that we know scope, we have a starting point to calculate the other things - but first we have to project rough cost of the scope.

I found the easiest way to do project management is to just walk the site and documentation that I've had to already create, and do time estimates on everything. There is a map of the entire site, with pages that explain what you need to do on each, and you've extracted all the common elements as well. What else do you need? Just put a rough time estimate on each "page" on how long it will take to complete. I usually measure this in days, with 1 day being the minimum, and with a week being a very complex page in WebObjects (that probably should have more of it's elements broken into sub-components). Do the same for the global components - and the interface scope is estimated.

You still have to factor in the costs for the model - but that is usually a smaller scope than the front end. EOF is very fast at building the model, and then mapping the model to EO's (entity objects). So most of the programming load in this area is in common helper functions and business logic (the middle layer). But both the back end, and the front end design should give you a pretty good idea of what is required there. Just write down all the things you need to do, give them a rough load estimate, and add them all up. Those routines can vary far more in time to create depending on your business logic, so I don't know of any hard and fast rules - I just tend to approximate based on fractions of a day, to days. But this will give you a working set of information on how far completed you are.

Now don't forget to pad the estimates - not because you are lazy or because you are trying to add in nap-time, but just because of the inevitable scope creep. There are going to be missteps and additional things added. Also the further you get in the development process, the more the programmers are spending time supporting the features (and bugs) that they've already created, and the less time they have at just pounding in new features. So remember that the rate of development starts fast, and continually slows down through the process. This gets worse once you've actually shipped - because now you have customer support and those communication issues to deal with - and of course the more bodies you add to the process, the more communication overhead there is (and things get slower as well).

The last thing to do is factor in all the final polish. Deployment is going to take time and resources. All the IS stuff, like setting up the machines, the servers, tweaking them, figuring out how to get everything talking, and so on, can be time consuming efforts. And then once you are ready to deploy there are other major efforts in profiling the App and figuring out where the bottlenecks are (or are going to be).

The total time estimate is just the count of all the time for all the functions. That should give you man days (or man months) of work - and you can use that to estimate how many people the project will take for a given time. But remember, time is quality. The more time the project has, the better the results. A few people on a project is good - it gives you a diversity of ideas - but beyond a few, the rate of development does not increase much. And there are latency versus throughput issues. Some things just have to be completed before others can be done - it is critical to look for these things. Nine women can't have a baby in one month. You want to get started on as many parts of the project as is possible as early as is possible.

Tip: Again, I tend to use the web to manage a web project. I put the time estimates on the web, often in an outline of the site itself - with total time and time towards completion (so far). Tracking how far you've come, and how far you have to go is pretty easy this way - and everyone can see progress and what is left.

Implementation

Implementation is where the rubber meets the road. It is where you have to do what you said you were going to do. This is much easier now that everyone has agreed on what it is you said you were going to do. You have the basics of a site map, where each page will be, and what is going to be on each page. You have the basics of what you have to store, and where that should be in the database. And you should have collected the basics of the common elements. You even have rough estimates on how much time each thing should take, so you can divide up the workload. Now get to work.

If you were really thinking ahead, you could have given yourself a head start. A trick here is that I use WebObjects to create most of the documentation and prototypes to get to this point, so that when it comes to implementation I have a working prototype and starter project.

When I create a site-map for each page and function - I do it using WebObjects itself. Thus the prototype is the documentation, and the prototype becomes the implementation. Just create a bunch of static HTML pages (using WebObjects), and use static links to take you from one page to the other. For the look, just include .gif or .jpeg snapshots of what each page will look like - and at the bottom of each HTML page, I just put the list of things that this page needs to do. When it becomes time to implement, I just snapshot this project, and I have a starter project with empty pages all ready to be filled in. As I implement each page I have a list of things (features) that need to be done, and I just remove from that list as I go. The documentation of what still has to be done can be right there (in the programmers face). You can even add in notes and issues to each page, for all those little things you don't want to forget (the developer notes on limitations of a function or page). And if you are fancy, with very little effort, you put these in a conditional so that you (developers or internal people) can show these notes are not - and you can keep the documentation and issues always in sync with the product.

The model work can be done the same way (on a smaller scale). I use EOModeler to collect a list of entities and attributes, and even show their relationships (sometimes). To create the documentation you can just generate the classes, and put in comments what each class or method needs to do (assuming you are putting business logic in the model, and not above it). Using Java Doc can be a big help here. When I'm done with the design phase, I can just tell EOModeler to generate the SQL to create the database.

As the design is maturing (as you go), you can even do other things. For global elements, create those elements (components) and what they need to do, and include them in or on the pages where they need to be. Then global elements are global in the documentation as well as in the design (and you just need to replace them with the functionality and they will work everywhere). For a list of support classes and middle layer business objects and methods, I just create place holders and shells using ProjectBuilder.

I was even thinking of getting fancy, and building a bug-tracking software right into the product itself (either the product or the shadow-product with the documentation) - so that QA could just turn on bugs using a back-door and add bugs right to the page that is having problems. And of course developers could view them and fix them right there as well. But there is a bit of work on that task (synchronization of pages and bugs, and good reporting abilities) - so I'm saving that one for a later effort.

There is still a whole lot of work to be done to finish the implementation- but at least these techniques can give you a have a head-start.

The goal is to make the documentation becomes the prototype which becomes the starting point for implementation. And later on the implementation can still contain parts of the documentation, and design notes, and other things. You can manage them in parallel, and keep the outline (with documentation, snapshots and project management, and bugs, etc.) separate, or put them in your actual project. There are strengths and weaknesses to both methods. On smaller projects I tend to keep them all in one, on larger ones, I've kept them in parallel - mostly because on larger projects I can afford to have someone else maintain the parallel effort, and it allows more diversification between versions. But the closer to the design you can keep the implementation (and vise versa), then the better you were at achieving your goals. Both ways result in getting a lot closer to your targets.

In-out, repeat if necessary...

Remember, if you complete this process, you have only completed one iteration. That isn't finished, that is just the end of the beginning. So go back to the top, start over, and repeat "if necessary" - and it is always necessary.

WebObjects is a Rapid Application development tool. So don't fight the tool - use it. It is for prototyping (as well as deployment), so use it for the prototypes. Websites are good for dynamic documentation. WebObjects is good for building websites - so use it if it makes sense for you. At least consider using HTML and a website for your documentation - eat your own dog food. This all may seem blindingly obvious, but I'm surprised by how many companies (and individuals) don't capture their goals in simple design documents, or how many don't use prototyping or online documentation.

Analysis and design don't have to be hard. Most of the academic books written on these subjects have good ideas in them , and good techniques - but can't be followed to the letter in the real world. Most of them are such overkill (for the commercial world) and so complex that people either waste too much time trying to do everything, they get frustrated by the books and all the things they have to know so they do nothing, or most often they try to do the former (and do everything) and burn out half way through and give up or say, "good enough" (and end up with nothing). Be pragmatic and use what works - for me, it has been starting at the top, and making a thin shell, and then filling it in and maturing it over time. In Aerospace, BioMed, or some financial institutions, there is so much risk to what programmers are doing, that they can afford huge time wasting processes and reduced development efficiency (and larger head-counts) for the sake of liability and documentation. Most commercial organizations can not afford that. Know your market, and what makes sense for your company and market.

Most companies try to separate development from project management, and apply abstract project management tools and layouts to a development task. It doesn't have to be that hard - just collect the requirements (scope), and layout how much time each thing will take, and add it up. You'll miss, but you'll usually be much closer than if you are basing your estimates on more abstract techniques. The closer you are to development (in the time estimation process) the closer you are likely to be to the target, and the better your estimations are likely to be.

It is amazing how many projects (and managers) still try to still do waterfall processes. They try to do all the design up front - as if we are omniscient beings - and they ignore how back-loaded the process is. You slow down as you go, because as you go you need to keep adding in more and more things - like documentation, support, bug fixing (and bug hunting), and so on. You also need to adapt to change as you go, and accept that feature creep is going to happen. Don't blame - try to capture the change request for a future version or reprioritize and trade-off functions - but accept change and that you can't think of everything before you start.

It takes many tries to get things right. Instead of trying to do it all right the first time, just accept the iterations and pick the low-hanging fruit. Layer the development (and analysis, and design) into iterations. This makes things simpler early on - and so you make quick progress (thus making everyone feel happy and productive). This also teaches everyone good habits (the process). The problems you think you're going to have may not materialize, the ones you are likely to have you weren't going to think of anyway. Don't design for features that may come down the pipe in years (or that you'll never have time to get to) - redesign when you need them, you'll know more at that time, and you'll have learned more during the time in between, so are likely to make better decisions later anyway. So by starting simple, you are training people on something easy, so that in later iterations, as things get more complex, everyone is trained and comfortable on the process. When later (tougher) problems come up, everyone is more skilled with the process, as well as understanding the solution - so they are more ready to respond to any problems that arise.

Lastly, by doing iterations, you always have the safety net of previous iteration to fall back on (if you are late to delivery). You not only have a more accurate way of mesauring how far you've come, how much is left to do, and how long each iteration takes - but you will have addressed the most important functions first, so they have the most time to mature. And while earlier versions may not be as full functional as the goals, they will be working versions which you can ship if need be - thus reducing risk to the company.

Doing things in layers (iterations) is faster, easier, makes everyone see progress and get more motivated. You are able to show others (investors or customers) the product sooner, and even sell it sooner. And it gives you more time to develop and mature the product - thus meaning a better product, for less time and less effort. Which is what the whole development process is about. Fortunately, WebObjects as a tool, can help make many steps of the process easier as well, if you use the tool for what it is good for.

David K. Every is a Senior Software Engineer who has done many enterprise and client-server (n-tiered) solutions, a WebObjects programmer, and the creator of the MacKiDo website (http://www.MacKiDo.com). You can reach him at dke@mackido.com.