Multidimensional Software Design – Do you agree?
Business problems solved by software systems are inherently multidimensional with dozens or sometimes hundreds of dimensions and complex relationships among them.
To name just a few basic dimensions: time, number of users, frequency of use, number of transactions processed per hour, transaction size (in bytes, money value, etc), business value of a feature, number of input fields, number of possible field values, network bandwidth, processor speed, uptime level required, hardware failure frequency, etc.
Thus a business problem has a shape/body in a multidimensional space (like a 3-D shape, but in more than three dimensions).
Software Analysis is about determining the multidimensional shape of the business problem and coming up with a streamlined model of it focused only on the significant dimensions and relationships.
Software Design is about coming up with a streamlined multidimensional solution shape to match the problem shape.
Multidimensional thinking is absolutely necessary in this process. Without it one can not envision the true shapes of the problems and solutions and can not separate the significant from the insignificant in them in order to arrive at streamline models.
Thinking that is only logical and linear is impotent when it comes to software design.
Yet, having decent logical thinking ability together with some mathematical ability is considered enough to enter the software development profession.
In the early days programs were more algorithmic, logical and linear than today, as they mostly implemented algorithms to solve scientific problems. Quickly they became much more complex and multidimensional and nowadays we think of them as “softwareâ€.
Many have attempted “software engineeringâ€, few have recognized that software development is inherently a DESIGN activity. It is so, because every business problem has its own unique shape in a complex multidimensional space and thus it tends to have an original optimal solution.
Someone once said that the difference between a great design and a good design is an order of magnitude. True multidimensional software design, indeed, typically results in an order of magnitude improvement in terms of project cost, amount of code, available functionality and quality. It is the real key to successful software projects.
There are many methodologies and frameworks aimed at guiding software development towards successful results and avoiding project failures. Not withstanding the many useful principles and rules they advocate, they all fail to address the most fundamental thing about software – its multidimensional nature!
For instance, the number one rule to follow in software projects is to include in the team a software designer with strong multidimensional thinking ability. Right there you have insured an order of magnitude better result. You would then also need someone with fast logic and good attention to detail to insure quality.
Unfortunately people with multidimensional thinking ability are a rare breed. Their number is much less than the number of software development teams out there and this is the main reason for many failed or more often overly expensive, poor quality, inflexible software systems.
Multidimensional design principles underlie most concepts in software development – Object Oriented, design patterns, n-tier and service oriented architectures, etc. etc. They are all solutions to certain multidimensional problem models. These problem models should really be well documented along with the respective solution models in all literature and teaching. Otherwise, as it is common today, many will tend to keep applying the same solutions even for problem shapes that are not anymore matching and arguing alternatives will be difficult.
P.S.
It seems that there are three nice categories of people in regards to spatial thinking - people with fast logical linear thinking (good at logic and algebra), people who in addition think well in 2-D (good at geometry), and people with multidimensional thinking ability (good at 3-D geometry and Physics).
I agree...
...but consider that we have no calculus for comparing bodies in hyperspace, and that managers, when they engage in "multidimensional thinking", use either wooly and adhoc forms of thought...or else barbarisms, clearly false, such as "bigger is better".
For example, the "comparision" of two TREES is the LABOR involved in CONVERTING the one to the other. And note that this is a projection operation, which maps the two trees (2 dimensional objects) onto two one-dimensional vectors, to which we can apply the simple less than, greater than, etc. operations since usually less labor is better than more (in fact, this could be a fundamental axiom of your calculus).
And you might be able to generalize this labor theory of value as a generalized comparision operations as long as you can persuade your client to let your authority be Karl Marx. If that's impossible try David Ricardo.
But typically, the analysts at companies with such visions as yours think in ways rigidly sructured by capitalist ideology and the resulting software is a mess, since literally speaking, they have been ideologically trained not to compare large systems in anything like a rational fashion, and instead look to what the boss "wants".
Deep Thought in programming runs against the grain of a society which relies on ignorance in the large, and this is why the employees of these "vision" companies quickly learn to not program and to write "white papers" instead, drained systematically of Deep Thought, or anything like precision.
The whiteness of the white papers is in fact ideological reassurance that the author is no boogie man and no troublemaker, and the deliberate vagueness at critical points (such as "how do I compare two tree structures") is reassurance to management that the SOCIAL change implicit in a disruptive TECHNICAL project will be nil.
In fact, everybody breathes a sigh of relief when the technical project runs off the rails!
Degrees of Freedom
I've always looked at it from the opposite perspective: when stating a new design there is a large number of degrees of freedom. You have the ability to build virtually any solution to the problem. As the analysis progresses, you uncover knowledge that quickly starts to limit the variables on their various axes. New constraints (restrictions), driven by the problem domain, the technical domain or even the development environment all limit the possibilities in the design. It happens quickly, so it often seems that the first 25% of the design process locks in 75% of the variables.
When you build a tool to solve the business needs, you can allow it to be flexible and dynamic by not locking in some of the variables, instead allowing them to be configurable. The more degrees of freedom, the more dynamic the solution. If you focus on what you know to be true and let the rest of the solution get driven from a configuration file, you can bypass a portion of the design (small but significant in effort), effectively solving it by correcting the dynamic parameters at runtime.
What about OLAP
As a DBA, I read this and just immediately thought: What about OLAP? It seems to me that, at least for reporting and Business Intelligence (whatever that means), we have the ability to output the data in as many dimensions as needed. People seem comfortable with input and functionality in a normal, "flat," two-dimension metaphor, so if we can produce the data in multiple dimensions for analysis, aren't we largely there.
Pardon me if I am just thinking in a 2-D way and not getting it.
Geometry of the problem space
I'm not sure that I completely agree with the multi-dimensional nature of the problem space (and solution space--if, in fact, there is a difference). I do like Paul Homer's comment regarding degrees of freedom. However, I've always found that my most efficient way of working was to think of the problem space as a large sheet of paper (say, a roadmap) that has already been folded up in a 'regular' fashion (i.e. not just crumpled) but is presented to me completely unfolded. In that case, the solution is already fully described by the problem. Keep in mind that most of my thinking regarding problem spaces has been shaped by all the time I spend with business databases.
My father taught me a very simple algorithm for refolding a map. A 'fold' is defined as a crease in the paper which is continuously peaked (or valleyed) from one edge to another. Identifying and applying those folds will result in a new 'plane' with new folds. Lather, rinse, repeat.
I find that manner of thinking to be very useful when working with databases. I try to start with a raw list of fully-qualified data points (customer name, customer address, order number, order date, etc.). I then group the information, thus creating 'folds' (to separate 'kinds' of information, like customer and order). To 'execute the fold', I separate them into their own 'planes' (tables) but maintain a connection via appropriate relational data. The ultimate objective is to end up with something can be 'unfolded' into its original form, just as the map must be unfolded into its orginal form.
I'm well aware that I've just described a basic and standard normalisation strategy. But I feel much more confident of my results when I think in terms of folding and unfolding. One of the great breakthroughs in my own thinking regarding this folding strategy came when I got a little puzzle made by Rubik (Rubik's Rings?) that allowed folds in strange ways so that you could actually change the shape of the puzzle and move around the various squares making it up. Playing with that little toy made me think about other ways of 'unfolding' my databases.
If the space were truly multi-dimensional, then I'd have to fight with projections and transformations. I'm quite sure that I'd strip my gears on that :) Maybe what I've described is already equivalent; maybe it's underpowered or restrictive in terms of what's possible with multi-dimensional thinking. But I don't go insane :)
(I love this board!!!)
Topology?
If software systems are objects in n-space then they have in the most general sense a topology. Topologically, a neatly folded map is equivalent to a sloppily folded map until the sloppy folds cause holes.
Software systems when viewed with modern graphics tools may actually "look" more like natural scenes than blueprints or schematics owing to their complexity. Perhaps we could "evaluate" software architectures by mapping them onto natural or city scenes, and then flying or walking about.
However, Dijkstra teaches us the necessary limitations of "visualization" by showing how the traditional visual method of proving that the square of the hypotenuse equals the sum of the squares of the opposing sides causes us to miss its generalization to nonright triangles.
Which returns us to the idea that software objects need to expose calculi unique to their architecture. Many "real world" problems, when viewed as a set of specific problems as the requirements necessarily change, turn out to be mastered by means of a formal language.
However, real users, as damaged by the conditions of their jobs, typically react negatively to formalization and want what they cannot have, that the software artifact provide results as vague as their own thought processes, a vagueness which is romanticised.
For example, a correct solution to a reinsurance (insurance of insurance companies against large claims) scam such that a dishonest broker has sold a circular chain of policies, each reinsuring the previous policy but sold at a higher rate, thereby making a profit, with the last fool in the chain being "insured" by the underwriter at the head of the chain, would be DETECTED by an adequate solution for automated reinsurance management.
You'd make a linear search of a stack. But my guess is that this would be rejected as a feature by many real users who in fact have made arrangements with their friends to look the other way when chains are arbitraged in this fashion.
And in a company dealing through its employees with such brokers, woe indeed to the lowly programmer whose software detects this as an unauthorized feature!
Reinsurance poses intellectual challenges. The problem at least in American business is that in many American (and, apparently, Chinese) companies the excessively laissez-faire, fuck-you attitude of the owners naturally filters down to the employees, and the latter make offline arrangements to support quiet frauds including cyclical reinsurance arbitrage. In these companies, programmers are too scared shitless to think.
Another example is Diebold, which owns the software that handed Ohio to President Bush in 2004. Word on the street is that the programmers in this company are silent and scared by continual investigations of their software.
My view is that efforts in the large to solve once and for all the grand problems of software engineering would, by eliminating an unknown quantity of high-level and low-level fraud, be resisted by the suits.
Software engineering properly understood is corporate governance and in a society with zero respect for the dignity of the working man or woman, software engineering will be a joke.


Multidimensional Software Design
This is very interesting, orlink. On the surface, this rings very true to me:
Would you mind providing an example of contrasting linear and multidimensional approaches to a particular problem? I don't think I have a clear picture in mind yet as to what "multidimensional thinking" is exactly. Is this something that's been formalized, like "systems thinking"? Is it a discipline that people are studying?
One problem I see with the term "multidimensional thinking" is that some company has trademarked it. Beyond that company's use of the term, Google does not turn up much (nor does Wikipedia).
Thanks again,
Dan