Now, of course, the vast bulk of that is pure junk. But that's not the point. The point here is that even as individuals, we're getting buried in our own computer files, from digital photos to ripped MP3s of old concerts, to word processor documents. How to keep it all safe? How to keep it all organized?
I was recently talking with my mother and my sister (both non-tech people) about their digital photos. Did they print them all off and put them in photo albums? No. Did they back them up in any way? Well, my sister dumps hers to a CD occasionally. But mostly, they just leave the photos floating around on their hard drives. I'm sure this is the case for millions of others out there. Even as digital photos spread virally through sites like Flickr and through simple emails, they'll be lost by the millions as hard drives crash or as that old directory with all those photos from 20 years ago is forgotten.
We haven't yet come to grips, as a society, with what it means to store so much of our vital stuff as zeros and ones on disks and flash cards here and there. Such stuff isn't really real, is it? It's easily lost, or even worse, stranded.
Think of your grandfather's slide projector. Remember sitting there, watching as oddly elongated and out of focus images of birthday parties and vacations to Mt. Rushmore were projected onto a bedsheet tacked to the wall? Who has a slide projector now? There are millions of boxes of, probably, billions of slides sitting in attics all around the world that no one has a means of viewing.
Except that they actually CAN be viewed. Sure, no one has an old-fashioned slide projector anymore, but you can hold a slide up to the light and see it reasonably well. You can still get slides printed as photographs if you want. Even if this technology goes away completely, a slide is still a real object with a real image imprinted on it. Some clever mechanic could build a slide projector with a box, a couple of lenses and a light bulb (or even a candle if it comes to that).
But what of those digital photos on the CD or the DVD or the hard drive. If computers go away completely, no on will ever be able to extract your vacation photos of yore.
Its unlikely that computers will go away altogether. Data stored and backed up on remote hard drives is probably as safe as anything for quite some time. But even if the data is out there, will you be able to read it?
I've begun to think about this in terms of my own personal writing. Like every other writer in the world, I use a word processor. But as I think about it, this is a risky thing to do. If, 20 or 30 years from now I decide to go back and work on a story I started yesterday, would I be able to even view the electronic file? Will word processors 30 years from now be backward compatable with MS Word 2003? Unlikely!
So here's what I'm thinking. Certain electronic formats are so raw and basic that they are unlikely ever to change. I'm talking about the WAVE file, the JPEG, the bitmap, and the ASCII text file.
How crazy would I be if I started doing all of my writing in a plain text editor? Actually, for almost all purposes, a modern word processor is overkill. Your story (or school paper) does not need a variable width font, bolded text, and 30 point headings. You can do everything you really need (write words, make paragraph breaks, save your work) in a text editor. Well, OK, spellcheck might come in handy.
If I took this approach, I'd have other advantages too. I use source control for all my writing. If I saved everything as plain text, I'd be able to do diffs on my writing files! Imagine how handy that would be.
I seem to have drifted into the subject of writing now, so I'll wrap it up. Someone tell me, am I crazy? How worried do we need to be about our data being stranded by shifting tides of technology?
]]>What I found so interesting about this post is that it advocates and speaks to the power of an approach that I had long ago dismissed as sloppy or even frightening--the passing of free-floating functions as method/function arguments.
At the bottom of the post, he links to an amusing takedown of Java as an overly-rigid OO language. The overall point seems to be that functional programming is powerful and leads to ways of thinking that OO does not. I can't speak to the affect of one programming language or another on the way people think, but I can give my impressions of the funtional aspects of what I call free-floating functions.
What I mean by a free-floating function is a function that is not tied to a class. For example:
doSomething(thingToDo);
as opposed to
ThingDoer.doSomething(thingToDo);
The thing I've never liked about languages that allow free-floating functions is that they lead to sloppiness and confusion. Where does the magic non-name-spaced function exist? How can I access it? What do I need to import/include to get to it? What happens if there are different doSomething() funcions I want to use?
To my mind, Java and other OO languages provide a simple means of organizing functions as static methods that reside in an un-instantiatable object. The benefit you get with this approach is namespacing. The drawback is a slightly longer call. Is that really such a big deal?
What Joel describes as particularly powerful is the ability to pass a function as an argument, even one created on the fly in-line. He asks if my programming language can do this. Well, my language is Java and the answer is yes and no.
No, you can't pass a function as an argument nor create one inline. But, yes, you can accomplish anything this practice can accoplish--you just have to use slightly different means. And, actually, I think the result is Java is more readable and maintainable than the result you'd get with a free-floating function.
To pass a "function" to a method in Java, all you need to do is make the function part of an class and pass an instance of that object. This is trivial for any Java programmer and most Java programmers do this sort of thing every day.
In fact, I can easily implement Joel's map() function in Java.
public interface ThingToDo {
public int doThingToInteger(int i);
}
public class Mapper {
//Makes it so you can't instantiate this class.
private Mapper(){}
public static void map(ThingToDo doer, int[] intArray) {
for (int i = 0; i < intArray.length; i++) {
intArray[i] = doer.doThingToInteger(intArray[i]);
}
}
}
//And then somewhere in a method we'll have this line:
Mapper.map(new ThingToDo() { public int doThingToInteger(int i) {return i*2;}; }, a);This is a trivial example (as Joel himself says), and yes, the inline class creation is ugly, though I would argue, not so much less ugly than inline function creation. A better practice would be to create the specific ThingToDo implementation as a separate .java file. Some would call this sort of thing clutter. I simply call it organization.
What the OO solution to this problem gives you, though, is some nice name-spacing, the potential for type safety in your map method and a potentially much more readable batch of code. What's the major issue with this?
I'll also admit that I'd never thought of farming looping itself off to a utility method. That's a terrific idea for all the reasons Joel argues. I'm just saying that there's no reason this can't be done in an OO language as easily as in a functional language.
]]>"You may leave your message after the tone. When you are finished, you may hang up, or press pound for more options. To leave a callback number, press six. To page this person, press star star." And so on.
This is an excellent example of software (because of course, voicemail systems are software) with two classic design mistakes that end up frustrating and wasting the time of most (let's guess 99.9% or more) users.
Mistake one is to let the novice user drive design to the extent that it imposes on sophisticated users. Almost all novice users of something as simple as voicemail will become sophisticated users within a matter of moments. I'm talking about the part of the message that informs me that I may hang up when I'm finished leaving my message. Not to rant and rave, but is there anyone with a telephone who has never encountered an answering machine before?
The second mistake is allowing seldom-used features to impose on everyday users. Here I'm talking about the instructions about leaving a callback number or paging this person. These are two features that I've never in my life used, and I doubt most people ever have. And if I were to understand why it might be necessary for me to leave a callback number (most cellphones capture the caller's number without them having to do anyting) or page my little brother (I'm calling him on his cellphone, after all--what's a page going to do?), then in all likelihood I would already know how do perform this task and would not need the helpful voice to instruct me every single time I leave a message.
This sounds like some dude ranting about his pet peeve. Well, OK. There's some of that here. But it gets worse. I assumed that, perhaps, this annoying tag message was a "feature" that could be turned on and off somehow. I called my own cell phone recently to see if I had a similar message, and, of course, I do. So I attempted to dig into my voice mail options to see if there was a way to configure this. There is not.
Even if there were, defaulting to the idiot/advanced features message is simply inexcusable software design. Think of the millions upon millions of hours that people around the world have wasted listening to instructions on how to perform the simplest task and then more instructions on how to use features they don't care about. Even if you calculate the cost of the wasted time using minimum wage, the amount would be staggering.
Windows XP, Word, and other consumer software are rife with such time-wasting and distracting features (paper clip and tail-wagging dog, I'm talking to you!). It's a tough balancing act, I know, to satisfy the most common users without scaring off new users.
But voicemail makers of the world, you are the worst offenders. I beg of you. Stop wasting our time. We are not idiots. No one has to tell us how to hang up the danged phone.
]]>I first heard about the possibility of sending internet data over power lines from an article in the Atlantic Monthly (subscription required to view the whole article). This sounded like just the thing to solve my problem. You see my house, while not huge is kind of sprawling. I've got a separate little building in the back where my office is and that's where the computer network was set up. But I've got a laptop and I always want to come in at night and have Internet access from the living room. I work from home, so I'm in that office all day and sometimes I just get sick of being out there.
Of course I set up a wireless network, and you can almost get a decent signal if you go all the way to the back of the house. But I guess these old homes are just too well constructed. From the living room you get nothing.
I was thinking of having ethernet cables run through the walls but this "HomePlug" thing sounded even better. (Actually, the HomePlug people have this vision of your power company eventually offering you internet service and maybe even telephone and cable TV through your powerlines. That's not happening yet, but it would be interesting.)
I did a little research and found that the Netgear model is highly recommended, and must say I was impressed. Just plug the little guys into the wall, plug in your ethernet cable and you are networked.
This is not a perfect system, of course. These devices generally will not work when plugged into a powerstrip, and the Netgear model is basically a huge wall wart, so it takes up some precious wall space. Also, the instructions say that you should only expect to connect maybe 5 or 6 of them in your home. But what this did for me was allow me to set up a wireless access point in my house and connect it very easily to my main router back in the office. Here I am now typing away on this entry in my living room.
]]>Well, let me make a prediction. As I write this, if you're working on a Struts application anywhere in the world and you're trying to start said application, this is having a big affect on you as well. You're frantically debugging, trying to figure out why your danged app won't start. What's going on?
It seems that the world of XML has some very sloppy practices built into it. These are sloppy practices used by developers all around the world, including myself, and they are a recipe for disaster.
We consumers and creators of XML files have been greatly encouraged to validate these files against DTDs. This is a standard practice. Any XML file worth its salt starts off with a reference to the DTD that defines its format. Typically these DTDs reside at a single universal location.
For example, the validator-rules.xml file currently giving me such fits references a DTD at the following location:
http://jakarta.apache.org/commons/dtds/validator_1_1_3.dtd
Can you see where this is headed?
Most XML parsers, it seems, when they encounter a reference to a DTD at a remote location take the trouble to actually read that DTD from that remote location. So what happens when that remote location is unreachable? Well, perhaps it depends on your XML parser, but the standard practice seems to be to fail in a very ugly way. In my case, the fact that jakarta.apache.org is down prevents my web application from even starting.
The practice of referencing remote DTD files is pervasive and, I've come to realize, stupid. When you reference a remote resource like that you're relying on some 3rd party to a) keep their web server up and running forever, b) keep that server at that address forever, c) keep that file in that exact location forever. Why are we relying on such foolish assumptions?
I believe that an XML file should never, under any circumstances, reference a remote DTD. DTD files are small and easy enough to package with the XML file itself. Thus, a reference to a URL can be changed to just name the DTD file.
This is a good practice for so many reasons. a) it removes external dependencies, b) it encournages developers to look at the actual DTD, and c) it means you'll no long have to care when jakarta.apache.org goes down.
]]>But I've found that most developers are not of this sort. Even I (as excellent as I am!) have a lot of trouble getting through a book of software patterns. Some of those patterns can be very hard to understand, let alone apply to a real world situation. What is needed, I think are a few simple guidelines that can help young, inexperienced, developers, or simply developers with lives, to write better code. If such a resource exists I do not know of it. While McConnell's Code Complete has a lot of great stuff, most coders are not going to slog through all 1,000 pages. Should they have to?
Dan long ago promised us a sort of Strunk & White for coders. Where is that precious book? I think we need it. [Ed note: I'll get it written someday!]
The Five Types of OO Classes
As sort of a primer on Object Oriented programming, I'd like to offer a few insights and opinions of my own.
After six years of experience in programming in an Object Oriented language, I've identified five different types of classes that occur in OO programming. Knowing and understanding the distinction and knowing how to build each of these types of classes is, I think, essential to writing decent code that is easy to read, understand, debug, and maintain. My five types of classes are:
1) Property Bags—the classic concept of an object. Basically a class that represents a real-world or virtual thing of some sort and in practice is basically a handy repository for a number of properties about that thing.
2) Thing Doers—A object that has a task to perform. If well designed, this task will be clear and concise.
3) Utility Method repositories—Such a class is often called Util or SomethingUtil. This is a repository of a number of common, static utility methods.
4) Plumbing Classes—Classes that are required not for the business rules, but for the implementation. Here I mean such things as servlets, EJBs, and so on. (Sorry to be Java-centric. I'm certain there are equivalents in other languages.)
5) UI Components—In fact, these could be considered a special case of Plumbing Classes, but they are so important and specialized that they deserve their own category.
I have created this list not as some sort of moot exercise in taxonomy but to make a larger point and that point is that each type has its use and misusing one or more of these types leads to sloppy spaghetti code.
Most developers use type 1 and types 3 through 5 with complete ease. Few developers ever write a class of type 2 and this is a darn shame because the Thing Doer is the most elegant of all of OO creations. Creating Thing Doers is an art I wish to share.
My Beloved Thing Doer
Far be it from me to be prescriptive (hey, it's my blog, though), but business logic should only and always be placed in Thing Doer classes. Oh, what a wonderful object-oriented world you build when you populate your code with Thing Doers. Let me see if I can explain.
Far too common is the placement of code into plumbing classes. A famous anti-pattern in Java is the monolithic servlet. This could also be the monolitic EJB. I'm sure .NET has equivelents to both of these constructs.
A naïve developer is given an assignment to create a bit of functionality. This developer knows she needs to implement the functionality in a J2EE server, so this means creating an EJB. So our young developer gets to work on here EJB and pumps it full of functionality, i.e., business logic. What's wrong with this approach?
Oh, so much I don't even know where to begin. But let me throw in here that I have done my share of such coding.
First of all, when you develop like this, you are tied to your implementation. If someone wants to migrate to a different system (Spring, perhaps) you have a whole lot of migration to do. Yes, you'll have to do some migration anyway, but you can save yourself a lot of potential trouble.
Even more important, though, is that code as described above is completely avoiding the strengths that OO can offer you. The strength that a Thing Doer object leverages is the module-level variable. If you've got a complex process with a number of variables that need to be shared, there is no need to pass every single one of them through to every method. You can make them module-level and each method can act on them.
No such approach is possible from inside of an EJB. Since an EJB is essentially a plumbing object, the developer has no control of its creation. It may be (almost certainly will be) shared. You can't use module level variables in an EJB or a servlet. So very many developers do not get this, but it is a great big no-no. Other threads will come along and change your values. This is a very dangerous practice.
To avoid this approach, many developers simply pass every single value needed from one method to another in the same class. Developers who do this fail to realize that a method call that does not access shared data in a module-level variable is essentially a static method call. Why not just put everything into a big Utility Method Repository and be done with it?
Instead of putting your business logic right there in the plumbing class, your plumbing class should instead do as little as possible. Mostly what it should do is instantiate and execute a Thing Doer class.
Let's say your EJB needs to create an invoice. Then your best approach is to start thinking about the design of a Thing Doer class called InvoiceCreator. InvoiceCreator might need to find an Order to go with the invoice. In that case, you might think about creating another class called OrderFinder.
What is left in your plumbing class is only what's necessary for the dispatching of your business logic to take place. In most cases this will be almost nothing, though it may be necessary to put validation or authentication code in your plumbing class.
I believe very strongly that all OO developers should learn to think in terms of creating Thing Doers. Such classes are easy to debug (no plumbing code, so no containers required), easy to retrofit to different implementations, easy to stitch together, to re-use, to write, debug, maintain and so on. I think such classes are the essence of OO coding.
On the Misuse of Property Bags
Another common approach (and I believe I will get some defenders of this here) is to put a lot of business code into a Property Bag class. To me this is a crude approach, but I'm prepared to admit that this is a personal preference. I find, though, that it can be hard to maintain an application with a lot of classes that look like property bags but which are laden with business logic. Should your Invoice class contain a createInvoice() method? Should your Order class contain a findOrder() method? I've found that once you start down that road, your property bag classes become huge and complex and difficult to maintain. Such an approach certainly violates the concept of cohesion.
There is much to be said for the elegance and simplicity of a property bag class that is nothing more than that. And the naming of a Thing Doer can make it's function clear. What clue will you have that the method for creating an invoice is in the Invoice class?
I believe that these are some simple and easy-to-apply concepts that can be a big help to OO developers. Arriving at these realizations was certainly a big help to me.
]]>Should we distinguish work from fun? I think not. Work, at its best, is truly rewarding and satisfying and even fun. Gardening on the weekends might be a hobby, but it is still work. In my twisted protestant-work-ethic saddled mind, the most truly fun endeavors are also work.
Life is work. What else is there? Watching television? If you're not struggling to accomplish something, whether it's raising a houseful of kids, building a dog house, or constructing the perfect diff algorithm, well, then, what's the point? Sure, hanging back with a beer and a few friends is good living, but is it really possible to enjoy that kind of relaxation if you haven't earned it, in your own mind at least, with a hard day's or night's work?
Maybe it's me . . . But I'm drifting off topic here.
Our jobs, as software developers, are made up of work and not work. The work is the good stuff--writing software, designing architecture, even doing maintenance code. The not work is what kills us. Going through elaborate processes. Attending long meetings. Filling out timesheets (yes, this can be labor in itself). Dealing with issue tracking software and other bureaucratic headaches.
Often the not work is necessary and important, if unproductive. When it takes up a reasonable amount of our time and allows us to spend 90% of our days on the work, well it's not so bad and resentments are few. But as projects grow and management gets more "serious" the not work grows as well, and then life at "work" (well for me anyway) grows wearying.
Currently, the project I'm working on requires me to participate in my client's process and software development methodology. This process is rather top heavy with not work. Every single code change requires a number of administrative steps to accomplish. I find that I spend perhaps 40% of my time on not work tasks.
Until last week, that is.
At that point our project entered the QA cycle and a whole new array of administrative overhead was introduced. In order to make a very simple code change to, say, fix a configuration file, here are the steps required:
0) If an issue does not exist in the issues tracking software, create one.
1) Check out the code from ClearCase (more on that monster in a moment). This step is fair enough.
2) Make the change.
3) Build the code with the new change (a process that generally takes about 5 to 7 minutes)
4) Verify with other developers that it is OK to deploy your build to the development server, there being no way to test locally.
5) Deploy your build (a process that takes about 5 minutes)
6) Test. Fair enough here.
7) Check in your changes. (Again, fair enough.)
8) Contact an administrative person to request that the stream be unlocked so your changes can be delivered.
9) If you had to create your own issue, contact an administrative person about setting your issue to "Open" status so that you can modify it.
10) Mark the issue as "needs peer review"
11) Find someone to peer review your changes. (A catch 22 here. This step is supposed to happen before changes are delivered, but no one can see your changes until they are delivered. Nice.)
12) Deliver your changes in ClearCase.
13) Go into the continuous build system and fire off a build (A process that takes 5 to 7 minutes)
14) Mark the issue as Fixed, being sure to include all of the details of what exactly you changed.
15) Hooray, you're done!
Accomplishing all 15 steps can typically take up to 2 days or even more, when the actual work part of the process takes maybe 30 to 45 minutes. When not work eclipses work to such a degree, software developers start to think about changing careers.
Now, to ClearCase, a central monster in this nightmare of inefficiency. Here we have a very expensive and supposedly powerful software development tool. A giant in the industry. (For those who don't know, ClearCase is a version control system that allows concurrent checkouts and has all kinds of nifty and complicated features). I find that, as a developer, I typically spend about 30% of my time asking ClearCase to do something and then waiting for it to do what I asked. This is a tool that costs $100,000 per developer? Really?
There are many, many UI problems with ClearCase, and there's the fact that it is slow, but I think the esesential problem is one of concept. The whole stream and concurrent checkout model is, so far as I have experienced it, at best useless and at worst counter productive.
Now I know there are legions of fans of concurrent checkout VCS's out there who love their CVS and Subversion. But here is how I have observed my own behavior under a concurrent system vs. my behavior under a single-checkout system.
I am more timid, more afraid to make changes under the concurrent system.
How strange. Isn't the whole point of concurrent checkout to make teams work more efficiently because developer X doesn't need to wait for developer Y to check in a file?
Well, in theory, yeah. In practice, a little different.
The most dreaded situation in ClearCase (and I would guess, in other concurrent checkout tools) is the merge conflict, when that horrible multi diff window pops up. You've got to sort out what you changed with what developer Y changed and you've got to get it right. This happens at checkin time, or even worse, at "rebase" time. You are not in development mode. You don't have the whole problem in your head at that moment. You're just trying to complete your process. And your diff tool is unlikely to give you the sort of help that your IDE does. Still, you've got to resolve those changes or be damned.
Your best bet is to contact the other developer and go through the conflicts together. If you're lucky, you'll get it right. If not, untested, buggy code gets check into your VCS.
How is this better than single checkout?
The advantage of single checkout is you get easy to understand prompts about who is doing what. If you can see that developer X is working on file A, you can call or email and say, hey, let me know when you're done. Or maybe you can work on this issue while you've got that file.
Basically, the point here is that concurrent checkout is no substitute for good communication within your team. And relying on tools to deal with these kinds of situations instead of communication is probably not such a good idea.
Ok, I'm off in rant city here. Forgive me. But the point that I wanted to make about ClearCase is that monolithic tools like this have a purpose and it is not the one developers have in mind. The purpose of a tool like this is decided not by the tool's users, but by the tools purchaser, i.e., management.
Management's goal in spending all that money on ClearCase and the like is not to make developers work more efficiently. Quite the opposite. It is to force developers to do all of their not work, to prevent developers from thinking, from making changes, from writing code, in short, from doing work.
]]>I am not uninterested in the outcome of this case because I am a BlackBerry user. I'm not the hugest fan of the technology--sending emails from a tiny keyboard has never really become a daily process for me. It's an oddly shaped cell phone as far as I'm concerned, but that's just me.
For weeks I'd heard about this case but did not really know what the issue was. Finally, someone wrote a comprehensive article on the subject--Tim Wu at Slate (check it out here). Anyone with an interest in the business of software development should read this excellent (if a bit irreverant, in the typical Slate style) article.
The underlying issue here is whether software should at all be patentable. In answer to the question that forms the subject line of this post, yes, you can patent software under the law as it stands today. Whether that is such a good thing is another issue altogether.
The basic gist of the suit is that NTP (actually just one guy) filed a patent some years ago for the concept of "wireless email." Did they create a wireless email device or any software to run it? No, they did not. They simply patented the concept before anyone else did. This is the basis of their suit. (Please read the article for the details.)
It might seem that this cas is not really about software patents--the guy could have just as easily patented some other concept, such as "vehicle that runs on 85% ethanol". The interesting thing about our patent system is that it does not require a working model for anything other than a perpetual motion machine. In every other case, what you patent is an idea.
Is the issue here especially applicable to software? I'm not sure. Software changes are typically about incremental improvements to existing systems--but so are changes in other industries. A slightly better engine, a slightly faster web browser.
Perhaps it is the entire patent system that is broken. Companies are patenting animals and fragments of the human genome. Does this make any sense at all?
Ben Franklin could be considered the founder of the open source movement, strange as that sounds. He was prolific researcher and inventor. As we all know, he invented bifocals and the Franklin stove. But he never made any money from his inventions. He didn't want to. He wanted them to improve peoples lives, so he did not patent them. Could this be a better way?
]]>Coders, of course, have more than one tool at their disposal. We have worlds of tools. But coders do like to code (see Andy's recent post if you don't believe me). It's what we like to think we do best.
Before I get too far along here, let me clarify what I mean by code. I argued for a broad definition of source code in a recent comment, and I'm not backing away from that really, but maybe clarifying a few terms. Let's say, for the sake of argument, that code is that which is written in a programming language such as BASIC, C, Java, and the rest. I won't attempt to define it further. And let's say that what I called "source code" in that other post might better be called simply "source." Only three paragraphs into my post and I've already gone off on a tangent. Typical.
The point I want to make here is that coders like to work in familiar territory. Present a coder with a problem and his or her mind will immediately go to work on solving that problem in the software language of his or her preference. For example, if you present me with a problem, I'll probably start right away with a possible solution using Java and some of my recent favorite technologies, such as Hibernate, SOAP, and maybe JMS. Java is my favorite hammer and I like to use it.
But many many problems are not best solved by building a mighty computer program, as I recently learned.
When our focus is on the coded solution we often forget the other tools in our toolbox. There's HTML, for example. Straight HTML without code is sometimes the right solution. There's also the application of an existing tool. If you want an easy-to-update website for internal company documentation, you'd be better off installing a wiki than trying to invent something.
And there's also good old fashioned, straight up SQL.
No, SQL is not code and if you're thinking of it as code then you're thinking of it the wrong way around. Code is a linear set of commands with logical and looping structures. SQL is all about sets.
Modern stored procedure languages such as PL/SQL and T-SQL allow developers to lose sight of this sometimes. It's fairly easy in these languages to create a cursor and loop through your data, apply logic, perform updates, and so on. If you approach SQL as a coder, this may be your first impulse as you approach any complex SQL problem. But when you do this you are really missing out on the true power of thinking in sets, something that can be very difficult to do, but can be incredibly powerful when done right.
I'm currently working on what might be considered a rules engine for a client of ours. We even call it a rules engine. It's not really a general-purpose rules engine, though. The overall function is to analyze data in a complex set of tables based on certain rules, and output the results to another table to give you an easy way of analyzing and reporting on the information you need.
My first impulse was to either create a solution using Java classes and Hibernate, or to look at an existing rules engine, such as JESS. However, for this application, neither solution was appropriate. We did not need a rules engine that could do anything. There were clear parameters of what our rules needed to do--look for information in a particular database about particular objects. And a solution in Java, especially one using Hibernate, would have simply been far too slow for the amount of data processing our engine would have to do.
It took me a while to come to grips with this last fact, but finally I did accept it. The objects that the rules will be applied to number in the hundreds of thousands, and growing. There will eventually be 300 or more rules. A Java approach would, of necessity, need to loop through every one of those hundreds of thousands of objects and apply every one of those 300+ rules. Processing would likely take days or even longer.
But a straight SQL solution lets us apply our rules without looping. This is possible because SQL is all about sets. By carefully constructing each rule as an INSERT SELECT FROM statement, we can harness the power of SQL to make our application run much much faster than would be possible with a coded solution.
Figuring out how to really make this work in SQL was difficult but rewarding. Coders are not generally trained to think in sets. We think in loops. Figuring out how to write a really complex SQL statement is profoundly difficult when you're used to thinking in loops. When and how to join, outer join, and so on is very tricky and, for me at least, takes a good bit of experimentation to get right.
In the end (well, it's still a work in progress, but the structure is there and the thing works) we ended up with a very neat application that is elegant and very quick, considering the vast amount of data we're processing. So I've learned that it is possible to create a very nice application using nothing but table structures and stored procedures. And I've learned that straight SQL can be neat stuff too. If Java is my favorite hammer, then I'll call SQL my new favorite wrench.
]]>I recently spent some time in Eclipse plugin land. I'd been avoiding this for a couple of years, because sometimes I just get tired of learning how to use new tools. Java development, especially, seems to call on you continuously to master some new best-of-breed framework (Struts, Hibernate, J2EE, JSTL, and so on). So a certain amount weariness had set in. But SourceJammer (my open-source VCS product) did not have an Eclipse plugin and users were constantly crying out for one. So finally I just broke down and got to work.
There is much to like in the Eclipse plugin architecture. There's a pretty nice framework that you use to extend Eclipse at various points (called "extension points") to, say, make a menu appear, or add your own preferences page to the preferences dialog. And building simple dialogs is a breeze (especially compared to building them using Swing). Overall I found the experience to be pretty easy. In just a few days of manic work, I had a working SourceJammer plugin (available for download at http://sourceforge.net/projects/sjextensions).
The main problem I had in figuring things out related to documentation. There's a pretty decent how-to doc built into the Eclipse help system, and some API docs, as well. But a lot of key details were missing. And though the how to doc refers again and again to an example project, there seems to be no way (at least that I could find) to view all of the code and config files for the example project. That's a pretty serious beef. But luckily I was able to find source code included in some other open-source projects out there and I was able to use these to guide me in what to do. And personally, that's the kind of documentation I generally prefer--someone else's working source code.
Working with an application that was built from the ground up to be plugged into is rather inspiring. You start to think that all applications should be built that way (I've heard Dan say similar things about Drupal, the software that runs this community site). Why shouldn't you be able to plug into and extend anything and everything. One of the big goals I have for the new SourceJammer client (which I hope to begin work on soon) is to make it much more pluggable. Something like that will take a complete re-architecting, though, and I must admit that I find the prospect of such a complete re-write to be a little, well, wearying.
]]>The reason I had to clean it up was not to fix the layout or to get rid of old content. The site is in pretty good shape as that goes. No, I had to spend almost two hours cleaning out several hundred spam comments all over the site, 100% of them for online poker.
Whatever you think about gambling (and personally, I can't stand it) I must insist that you agree that online poker is absolutely evil and anyone who participates in it is a first class chump. That's not just out of vengance. Well, maybe a little.
Your better blog and community website software (such as the software that runs the developer.* community site) provides tricky ways for real people to post comments while keeping the spammers out. That's great. The little magic image thing that Drupal uses is very cool.
But of course, it's a real running-to-stand-still situation. While the CMS designers are thinking of new ways to thwart them, the spammers are thinking of new ways to avoid being thwarted. Eventually, they'll find a way to interpret those images and Dan will have to disallow comments from non-registered users. And eventually even that won't work. The spammers will build an automated tool to register under a bunch of different names and respond from a bunch of different email addresses.
As I sat here at my desk deleting comment after comment, my mind had plenty of time to drift and think about other things. Of course one of the things I thought about was the fact that spam protection was non existent in my blog tool (Greymatter), which is too bad, because in many ways, I love that tool. I love the admin interface. I love how easy it was to install. I love how the files are stored on the file system so you don't need to set up a database. And I love how it generates real, static pages so you don't have to do any thing tricky to get your pages to show up on Google.
But with no spam protection, I knew I had to switch tools. I'm now using the freeware version of a tool called pMachine, and I'm pretty happy with that so far.
The larger issue that I stared to think about, though, was the things we do to make our living--the ethical and moral compromises so many of us make in order to earn our pay. Most of us in the software development industry work (either directly or indirectly) for big business or for the government. For the most part (and there certainly are exceptions) we are not engaging in unbridled public good in our work activities. We are not curing diseases or sending space ships to the moon, or reducing pollution. Of course there are software developers that do all of those things. But most of us engage in much more mundane activities--building financial reports, customer tracking software, and data entry systems.
I've never been asked to do anything as a software developer that I had serious moral problems with, but I've never been asked to do anything that made me feel like I was bettering the planet either. That's somewhat better than the work I did before I was a software developer, when I wrote educational materials for the insurance industry. Not evil in itself, but I was part of an overall infrastructure that was furthering ends that benefited that industry at the expense of the public (though of course we would never have put it that way). Not evil stuff either. But a bit murky.
I can't imgaine developing software for the millitary to help in the killing of other human beings. I'm sure it would be neat stuff, and possibly important by some measures as well. But I don't know that I'd want my hands dirty in that way, though I can understand how many people would have no problem at all doing that work.
But here's where I'm going with this and here's how it relates back to spam. I said earlier that the spammers are always working to get around the latest anti-spam tricks. But of course it's not the spammers themselves who do this. It's the programmers they hire! All of those ridiculous messages about online poker that I deleted from my site were all put there by some sort of a spam bot. The spam bot is a piece of software, so by definition it was created by as software developer!
I am stunned by this obvious revalation. There are software developers out there making their living thinking of neat ways to deliver ads for online poker, penis enlargment cream, herbal viagra, and hot teen porn sites. Some of them might even read this site! I hope they do, because I have a message for you guys and gals. You are doing evil work. The fruits of your labor hurt other people, invade their space, waste their time, hog bandwidth, and steal space on servers. Find another line of work. To paraphrase Jon Stewart, "Stop hurting the world."
]]>I hope I don't come off as someone who just wants to crack on Web Objects, but the more I work with it, the less there is to like. I think my essential problem is with the "one-tool-fits-all" approach behind Web Objects. As I explained earlier, WO is basically a case tool for web development. And like all such tools, it requires developers to do everything in exactly one way, using proprietary tools, while hiding all kinds of relavent information.
My goal has been to create a very simple application. I have a single table called User. All I want my application to do is 1) display a list of all Users in the system on one page, and 2) allow me to add, edit, and delete Users on another page. To mix things up a little, I added a reflexive relationship on the User table. Each User has a best friend, which is also a User. Thus, I want to be able to select each User's best friend from a dropdown list of all other Users. Simple stuff, I think.
Creating the first page, a listing of all users, was fairly easy to do. A wizard basically did the whole thing for me. All right. So far so good. But the trouble started when I started trying to figure out how WO binds my edit user page with a User object, and how I can tell my page which object to bind to. WO seems to want to keep this stuff in the realm of magic, and I don't like that.
What do I mean by "magic"? Well, when you create a new "Web Component" in WO (which, aparently, can either be a full page or part of a page), WO generates four files four you:
1) The "html" file, which looks more or less like HTML, but with special WebObjects tags included as pointers to where your dynamic data is going to go.
2) The "wod" file, which is a mapping file that defines the dynamic content and does some namespace mapping that is picked up by the WebObjects tags in the "html" file.
3) The "java" file, which, I think, represents the actual Java code behind your Web Component. I'm not yet sure how this guy fits in, but you can add methods to it that can be called from your "html" file.
4) The super secret "woo" file.
My first question upon creating my Web Component was, how does this thing know where to get my data? The java file contains a generic object called WODisplayGroup that contains all of my user objects. But there's no indication anywhere of when and how this is loaded with User objects. Well, there is an indication, but it is difficult to decipher. There's an auto-generated comment that says:
// This WODisplayGroup is initialized from the .woo archive in the component's constructor method
OK. What in the heck is a .woo archive? There are no .woo files visible in the ProjectBuilder. I thought this might be defined in some kind of properties window for the Web Component. If it is, I have not yet seen this. I did finally discover the .woo file on the filesystem, and sure enough, there are mappings to my User object in here. Mystery solved. I understand that maybe I shouldn't modify this file directory, but, as a developer, I hate these kinds of secrets.
I finally figured out how to get an edit page to load a particular user. At least I think I have. I had to create an "action" method in the user list page. This method returns my next page, the class automatically generated as a .java file, that is. The code looks like this:
public editUser2 editUser(){
editUser2 nextPage = (editUser2)pageWithName("editUser2");
nextPage.setUser(user);
return nextPage;
}Note that my edit user page is actually called "editUser2". That's because I couldn't figure out how to delete the original editUser page. (Side note: I finally figured out how to delete. "Remove Files..." under the "Project" menu. How could I have been such a fool?)
The first line of the method and the last line are pretty straightforward stuff. You pull in the "page" object from some kind of directory service. Then you return the page. The line I have trouble with is the second. This is a call to the setUser() method that I created, to set the User object for my page. That part I don't have a problem with. The problem I have is, where does the variable "user" come from? This is a member variable of the adminUser object, but I don't understand how I know it's going to be set to the proper user at the time the user clicks on my "action" link.
Now I'm in the process of trying to build and deploy my little app to see if any of this stuff actually works.
Well, I suppose I'm just going to have to trust in WebObjects. Oh, boy.
]]>In many ways, Java makes an excellent language for developing desktop GUI applications. The Swing toolkit is rich and versital and makes performing certain kinds of tasks relatively easy. (For example, mouse scroll button support is "automagic" in all JScrollPanes, and the default implementation--whatever you mouse over you scroll--is a feature I wish other platforms would embrace.)
But, of course, its limitation are legion, and it is certainly not your first choice when your goal is to develop a desktop GUI app rapidly.
In a previous post I mocked WebObjects for it's amaturish Java GUI. But anyone who has ever used Eclipse knows that Java can build rich, clean, and rock-solid visual applications. (Yes, I know, Eclipse does not use Swing, but that's another story.) And dare I toot my own horn (Nilges does it all the time, so why not?) and mention that the aspect of SourceJammer (the version control application I developed) that users typically say they like most is the clean and easy-to-use GUI. So we know it's possible to GUI development right with Java.
But out of the box, Swing has some serious limitation that make the learning curve steep and development slow and treacherous. One glaring problem is the lack of a standard combo box component with typeahead support. If you want typehead in your combobox in a Swing app, you have to either a) figure out how to implement this yourself, b) pay money for some 3rd-party toolkit or c) trust in one of the very few open source solutions (PSwing at http://pswing.sourceforge.net, which I also developed, being one).
Another glaring shortcoming of the Swing api is very poor support for printing.
Well, let me rephrase that. The support is excellent and highly configurable. It's just incredibly difficult to use.
For the past several days I was tasked with figuring out how to perform batch printing from a Swing application. More specifically, the application needed to hit a series of web pages and print each page--not the raw HTML, of course, but the generated page. This was far more difficult than it should have been.
The Swing JEditorPane provides reasonable rendering of web pages. Don't expect it to work if you've got frames or javascript, though. Low tech pages only, please. Regardless, that part was easy enough to get up and running.
It was the printing tha really gave me fits.
Explanations of how to print using Swing are few and far between on the internet. One of the better ones was located here.
In order to print the contents of a Swing component, you first need to create an Object that implements the Printable interface. There is no default implementation of Printable out there in Javaland that I could find. Nope. You've got to build your own, and this means implementing the print() method which needs to deal with the underlying Graphics object (arrggg!) and also figuring out which portion of the component to print based on the page number.
I must be kidding or simply mis-informed, right? Surely it can't be that complex? Graphics math? Apple II-e, anyone? Sadly, Sun's assumption seems to be that, yes, developers should implement the print() method themselves. See their tutorial on the subject here.
Luckily, I discovered a some code on a forum at java.sun.com that made this process a little bit easier. I combined the code posted on that forum with the code from the tutorial above (not the Sun tutorial, the one from www.ap1.jhu.edu). In the public interest, I post the combination here in the hope that this will be helpful to someone some day.
The following code can be compiled into a class that can easily print the multi-page contents of a Swing component. If fact, if you just use the static print() method, you're user will see the print dialog and everything else will happen without you having to get your fingers dirty.
/*
* Copied from this tutorial:
*
* http://www.apl.jhu.edu/~hall/java/Swing-Tutorial/Swing-Tutorial-Printing...
*
* And also from a post on the forums at java.swing.com. My apologies that do not have
* a link to that post, by my hat goes off to the poster because he/she figured out the
* sticky problem of paging properly when printing a Swing component.
*/
import java.awt.*;
import javax.swing.*;
import java.awt.print.*;
public class PrintUtilities implements Printable {
private Component componentToBePrinted;
public static void printComponent(Component c) {
new PrintUtilities(c).print();
}
public PrintUtilities(Component componentToBePrinted) {
this.componentToBePrinted = componentToBePrinted;
}
public void print() {
PrinterJob printJob = PrinterJob.getPrinterJob();
printJob.setPrintable(this);
if (printJob.printDialog())
try {
System.out.println("Calling PrintJob.print()");
printJob.print();
System.out.println("End PrintJob.print()");
}
catch (PrinterException pe) {
System.out.println("Error printing: " + pe);
}
}
public int print(Graphics g, PageFormat pf, int pageIndex) {
int response = NO_SUCH_PAGE;
Graphics2D g2 = (Graphics2D) g;
// for faster printing, turn off double buffering
disableDoubleBuffering(componentToBePrinted);
Dimension d = componentToBePrinted.getSize(); //get size of document
double panelWidth = d.width; //width in pixels
double panelHeight = d.height; //height in pixels
double pageHeight = pf.getImageableHeight(); //height of printer page
double pageWidth = pf.getImageableWidth(); //width of printer page
double scale = pageWidth / panelWidth;
int totalNumPages = (int) Math.ceil(scale * panelHeight / pageHeight);
// make sure not print empty pages
if (pageIndex >= totalNumPages) {
response = NO_SUCH_PAGE;
}
else {
// shift Graphic to line up with beginning of print-imageable region
g2.translate(pf.getImageableX(), pf.getImageableY());
// shift Graphic to line up with beginning of next page to print
g2.translate(0f, -pageIndex * pageHeight);
// scale the page so the width fits...
g2.scale(scale, scale);
componentToBePrinted.paint(g2); //repaint the page for printing
enableDoubleBuffering(componentToBePrinted);
response = Printable.PAGE_EXISTS;
}
return response;
}
public static void disableDoubleBuffering(Component c) {
RepaintManager currentManager = RepaintManager.currentManager(c);
currentManager.setDoubleBufferingEnabled(false);
}
public static void enableDoubleBuffering(Component c) {
RepaintManager currentManager = RepaintManager.currentManager(c);
currentManager.setDoubleBufferingEnabled(true);
}
}
If you encounter trouble using this class, let me know. And if it helps you, let me know as well.
]]>It was at a Christmas party in Atlanta last year that my good friend Trevor Conn (I always want to spell it Kahn. Kaaaaahhhhhnnnnnnn!) started telling me about a new software development project he was working on. It was to be a web-based application developed using Java/JSP. Ahh. My bread and butter.
Trevor's background is more on the Microsoft side of the fence, so there was a bit of a paradigm shift for him. I'm not talking about syntax or event models or software frameworks. That stuff is pretty easy to figure out, and genereally there's an equilivent concept somewhere under the hood, only with a different name. In fact, there is a much deeper division that separates the Microsoft world from the Java world.
I'm talking about money.
Trevor was surprised to find out that absolutely everything he needed to do Java/JSP development was available for free. And we're not jut talking about a couple of clunky compilers that required you to do your coding in notepad. All of the best of breed development and deployment tools you could ever need if you're doing Java development are out there on the internet for the taking. And with some free support and maintenance thrown in for the bargain.
While there may be some free development tools for the Microsoft world (nAnt comes to mind) for the most part, people who develop in that world expect to, well, get paid.
Trevor was genuinely curious about this. How can all these tools be out there for no cost? As the lead developer of a free, open source project myself, I'm in a better position than many to answer that question.
Really, the answer is not all that interesting. The short answer is that, with Microsoft so dominating the game, the only price at which many products can compete is zero dollars and zero cents. For example, no one is going to pay for an operating system not made by Microsoft. Ok. Ok. I know, there are literally thousands of Mac users out there proving me wrong everyday on that one. Fine. But the point stands. The only price at which Linux competes with Windows XP is $0.00. The same is true for development tools as well.
What you have in the Java world is a number of big companies that need to compete wtih Microsoft. They fund many of the big free, open-source projects in order to a) keep the Java development community alive and b) to provide some kind of competition to Visual Studio. Thus, IBM funds Eclipse and Sun gives away the JDK and NetBeans and all kinds of other stuff. And I believe (though I've not done the research to back it up) a lot of the big boys are funding the Apache project for similar reasons.
But in addition to the high profile projects funded by big corporate dollars, you've got the little guys like my baby, SourceJammer. Why would I spend thousands of hours of my time developing a source control system just to give away? Well, ego certainly plays a part. It's a neat thing to have thousands of people out there using and depending on your software. And you can dream that maybe, just maybe you've developed the next JBoss or JEdit that will become a quasi standard in the community. This would not only be good for your ego, but also for your career, especially if the work you do for money is a bit less cutting edge and rewarding.
And there's a karma thing to it as well. Giving back what you've taken. Because I use free, open source software every single working day. From Tomcat to Eclipse to Ant to Log4J, my work depends on free software. In fact, I am so used to development tools being free that, from time to time, when I go looking for an existing solution to a problem that I must solve, if I find something that is not free I immediately get annoyed and dismiss it out of hand. How dare they charge for software!
I'll admit that this is an odd attitude, but it's not uncommon in the Java development community, I think. Something would have to be awfully important for me to ask my boss to actually pay for a license.
Now, if the world were turned around, would the Java community still be the same way? If Sun or IBM were somehow dominant and Microsoft were scrambling to keep up, would all of these great Java tools still be free? Would people like me who come up with Java solutions immediately assume that our work should be given away? Somehow I doubt it. I think the culture of free software developed as a sort of rebellion against the culture of monoply. The altruism involved, as with so much altruism, when you really dig down into it, is triggered by a selfish impulse. What it comes down to is survival.
]]>Sometimes this is all good stuff. For example, last year, as part of a new project, we started using Hibernate and I got to learn some valuable stuff about using a new, cutting-edge, OR mapping tool.
But sometimes this is not so much fun. Right now, I'm teaching myself to use an Apple technology called WebObjects, which is what the WO in my title refers to. (In full disclosure, I've copied the title of this entry from an old article I found here.)
In theory, WebObjects is neat stuff. It's kind of like a case tool for building webapps. I'm not a big fan of case tools, personally, but there is a need to simplify web development, to free developers from repetitive "monkey code" tasks, and to create a paradigm for web development that seamlessly connects your data source to your presentation to your business logic, but without coupling the three. WebObjects provides one solution to this delima.
However, the implementation leaves something to be desired. I've only just started fooling around with this tool and my plan is to post my ongoing reactions and evolving ideas to this blog, in case anyone cares.
My first impression of WebObjects was that I'd never heard of it before I learned that I would need to start working with it to help a client on an existing app. I did some internet research and, although there is info about WebObjects on Apple's site, there's not much in the general world. And what is out there is mostly very old in Internet time--two to three years or older. So right off the bat I have two big concerns: 1) is Apple still suporting WO and do they plan future releases? And 2) is anyone else out there using WO? To combine the two--is WO a ghetto?
(To clarify one point--WO is cross-platform and uses Java. It's not an Apple-specific product. It's just created and sold by Apple.)
A developer at the client demo-ed some WO stuff to me and overall I was impressed with how it allows you to do OR mapping visually, connect an object to a web component, and then create an action for that web component to fairly easily display, create, and update data without doing a lot of obj1.setValue(obj2.getValue()) type code, or even worse, obj.setValue(request.getParamenter("value)) type stuff.
But as I've installed and started working with the thing, I must say that it leaves something to be desired, from what I've seen so far.
Issue 1: The installation required me to install Java 1.3 even though I already had Java 1.4 installed. What is this, Oracle?
Issue 2: The GUI development tools are very ugly and buggy. Again, what is this, Oracle? For one thing, it is possible to close an app's main window when you've got a dialog for that app open. That is just flat out ridiculous! And the gui looks like some very poor java 1.2 era awt junk. It is apps like this that give Java gui apps a bad name. Again, what is this, Oracle?
Issue 3: The OR mapping tool (called EOModeler) is not very easily configurable. Something like this, by definition, needs to work with multiple databases. But in order to get it to communicate with MS SQL Server, I had to a) copy the sql server jdbc driver jars into a special directory, b) modify a class path variable in a config file to point that directory and c) figure out how to do that more or less on my own since there was no help or documentation available that explained this process. Very very very very bad. Also, I had to type in my jdbc connection url by hand. Yeah, I'm used to that from building config files for my apps, but come on, people! There only maybe five or six major database platforms. How about a little wizard for each one so I don't have to figure out your whole oddball system in order to build a little Hello World demo app!
Issue 4: Project Builder--the sort of main clearing house development app in the suite--has a very odd way of creating a new project. You are asked to browse to a directory to create your new project, but no indication is made of what to do next. What it wants you to do is, browse to a parent directory, then enter the name for your project in the file browser dialog. PB will then create a new directory with that name and put your project in there. Thanks for explaining that. What if you want to put your project in an existing directory? Can't do it.
Issue 5: When you create a project, there are lots of different types of things in there. You get a big list of the types: Web Components, Resources, Subprojects, Web Server Resources, and on and on. You've got to work within this paradigm, it seems.
Issue 6: Suppose you create a Web Component, but decide it's junk and want to delete it. Too bad! There's no delete functionality whatsoever, as far as I can see. I tried deleting the directory it created on filesys for the component I wanted to delete. Then I restared PB. But that didn't help. Now the "Web Component" listing just shows the deleted thing in red. When I try to add a new one with the same name it won't let me. Is this a feature?
On the plus side, it did give me a wizard that very quickly built a display page to display all of the records in a table. It seems that it can do the same thing for forms and other components. My point here is not to bitch about how bad WO is (though I seem to be doing that) but to present a diary of my experience. Hopefully Monday, as I continue with my little demo app, I'll have some nice things to say.
]]>