I agree with the points you raised in your most recent article. The only thing I would add (actually expand on) is that managing client expectations is the key part of the entire process. Without a progressive education of the clients in software engineering processes, (change request processes, etc.), the project will fail. By fail, I mean as far as the customer is concerned. It can still be right technically, and maybe even commercially, but if the client is not happy at the end of the process, then you still have to look at the project as a failure.
The last four projects for which I have been team leader on have failed. In every case this was directly due to the inability of the project manager to manage the client's expectations or the client giving us incorrect information. The most important thing we have to always remember as software engineers is that clients, like most non-technical personnel, subconsciously seem to believe that computers are some kind of magic and fundamental changes can be done overnight ("It can't be that hard - it's just another field surely ...?"). They get this from the media and from the cowboy WYSIWYG developers they've encountered on previous projects.
Only a complete idiot would get a house built and on the last day of construction approach the builder/architect and say "Can you just add on another story - um over the kitchen I think. Oh, and take out the bathroom-we don't need one anymore." Yet this is exactly the type of things clients will do. Often this is because the business rules have changed throughout the development cycle. Often it's because of miscommunication at project inception. And often they're just ignorant and signed-off on the requirements specification without reading it (don't laugh it's happened to me twice now).
Finally be very, very careful hiring your technical people. Software engineering one of very few industries where people can make six figure incomes from just "picking it up along the way" (Imagine lawyers, brain surgeons or commercial pilots who just "picked it up"!). That is, probably around 40% of programmers out there actually are qualified computer scientists or equivalently trained/experienced. Hiring bad technical people will kill anything you ever do and cost you a lot of money-if it doesn't collapse your business.
The best thing you can do is 1) hold a detailed technical interview with applicants in the relevant areas, and 2) have detailed code reviews at regular stages throughout the development cycle (the team leader should do this). FrontPage users might get through the interview, but when they actually fail to contribute anything of any value, at any stage in the project-ever-you can then get rid of them, hire someone useful, and your project is still recoverable.
Dan, I could also keep going about these issues as I obviously feel very strongly about them. As humans we need closure, and we need a sense of progressive achievement. Constant failure due to avoidable incompetence will destroy the morale of any team. Have regular deliverables so the team gets that sense of achievement. Doing things the correct way without compromise will produce the best software.
All we have to do now is convince the commercial people...