We have a distinguished guest here at Developer Dot Star, apparently
The recent comment "Dijkstra" is from the famous maven of applied structured programming Edward Yourdon. Although this is "not verified," the style, courtesy and literacy are the Yourdon I knew.
For all you young kids, Ed Yourdon formed a company in the 1970s to teach the structured techniques in a real programming language after Mr. F. T. Baker of IBM found that their consistent use in PL/I had created an on-time, effective system (still in use) at the New York Times for journalistic information recording and tracking.
Yourdon's classes showed clearly how to write Cobol, the most popular business language of the time, in a structured and readable way. In Cobol you could do this with nested Ifs, and and or Boolean operators, and use of Perform (a primitive mechanism for subroutines). Although all data in a Cobol program was global to the program in a serious design error on the part of the 1960 developers of Cobol, you could use naming conventions to show the intent to localize.
I learned from Edward Yourdon how to write pretty good Cobol. By 1980, I was able at Baxter-Travenol to write a five-way match merge for personnel which unified its profit sharing and pension data streams in 24 hours, making my boss Joanne, a wonderful lady, very happy.
The only problem is that there has always been an unwillingness, through misguided desires not to hurt feelings, to discuss the larger social context in which all this bit twiddling occurs.
We know that we're working hard for a company such as Starbuck's, and we rationalize, as I rationalized, that we have kids, or need medical insurance for an aging parent. And, truth be told, companies run by a single compassionate man who likes to build boats are almost nonexistent.
Even in my current job, for that matter, I am giving kids whose parents have money a leg up on their assignments while most kids in Hong Kong have to struggle at the library at Tin Hau.
Where I drew the line was when the social facts prevented me from doing a good job: I'll never forget the "structured walkthrough" at Montgomery Ward, where the manager sat down with us and said "I know your structured programming teacher said 'no managers in structured walkthroughs': too bad."
I'll never forget the ex-Navy officer who was assigned to a late project at Motorola with the mission of finding trouble-makers and who would get in their face about their late code, telling them in so many words that they were swabbies and grunts, nothing more.
Weinberg, who worked with Yourdon, wrote about the ways in which this rage to control damaged the application of structured techniques. If you've been told you are shit, why worry about being structured or object oriented?
Hell, make a mess that seems to work or actually does, and leave it as a legacy to the company. More than one programmer has done this.
I've never done so: yet the paradox is that alienated programmers can and do see what I think is good code and see bad code. In C, I use Hungarian notation and instead of using i as an index, I use intIndex. This enrages programmers who have to code and maintain gargantuan C source programs: it's a waste of time.
Thanks to Dijsktra and to Yourdon, the 1970s were in actuality a time of rapid progress in software, much more rapid than the HTML-ridden 1990s. Yet this cannot be admitted by the powers that be, because the 1970s were also an era in which programmers, as working folks, were less controlled and less monitored and less alienated as a result.
Yes, Edward, I agree that Dijkstra showed us that as responsible applied scientists, we have to prove the correctness of large software systems. The trouble is that a correctness proof of Enron's software would have revealed that it was in part a tool for hiding losses.
The trouble is that a correctness proof of banking software for large and successful banks would show a systematic exploitation, such as occurs in my experience at The Hong Kong Shanghai Bank, of the small saver, the sort of person who *en masse* made the big bank big, in favor of the "premiere" customer.
The trouble in 1962 was that a correctness proof of the United States' new air defense SAGE system wasn't available and could not be available because it was classified. You see, Dijkstra's notion of proof, like that of any decent mathematician, was founded on the fact that ANYONE could see, from Euclid's proofs, that they were true, and a proof cannot for this reason make any reference to secrets.
The trouble today is that a Constitutional correctness proof of the algorithms that may or may not have been applied to aggregate phone data by the US to monitor domestic calls can never show that this data hasn't been used to monitor individuals. I was able to reconstruct calls from atomic phone hardware events, so no Euclidean "proof" can be produced by the administration that the surveillance is aggregate-only.
In other words, there was a real contradiction between the *glasnost*, the openness, of the structured walkthrough, in which people would admit their mistakes and point out other's errors because they weren't, in the 1970s, afraid for their jobs and health insurance, and the natural bias in American business towards shedding workers at-will. Weinberg's and DeMarco's personal solution was to more or less drop out as a lifestyle choice, and accentuate the soft side. I'd say that this shouldn't have to be a lifestyle choice, and that people deserve openness and transparency in daily life.
The Edwards
I'm following this saga with great interest, feeling pretty superficial blogging about an Orlando developer conference when such weightier topics are being discussed.
Enjoy Orlando, Donna
I have a blast when I get to go to VSLive. There is a certain energy when geeks get together and out of the cube. I've enjoyed my VSLives and before that my Share conferences for IBM mainframe geeks at which the evenings usually concluded with a drunken chorus of "we all live in a virtual machine", to the tune of Yellow Submarine.
Is Bill Gates there?
I hope I don't scare Edward Yourdon away.
Dazed and Confused
Some people may have thought Dijkstra was just kidding when he predicted that large organizations would collapse owing to the unmastered complexity of their data systems. He wasn't, and one example of unmastered complexity would be the US election system.
See http://www.freepress.org/departments/display/19/2006/1964: there is doubt that the US had a valid election in 2000 and 2004 because of security flaws in electronic voting machines as used in decisive states such as Florida and Ohio.
But how is this related to structured programming, of all things?
An unstructured program might be fully correct in fact: many authors of "spaghetti" code were persuaded of this and some of them were right (me, for instance, before I read F. T. Baker's 1972 article on structured programming). SOME programs worked in the interval 1950..1970 prior to the introduction of the structured techniques. Hell, my father checked each bank statement by hand and he assures me the bank was right even after it computerized.
But part of the problem was that the organization depended for assurance on the reliability of this primitive software on the programmers themselves, and the correctness of the software could not be proven, whether using formal methods or English.
Structured programming not only led to provable programs in an informal sense (we can discern correctness or its lack in structured code as opposed to go-to code) it also had the social impact of making programmers more accountable to management, and management accountable to programmers for a brief interval.
Macho epistemology, however, resisted the structured techniques because in macho (typically male) epistemology, one knows what one knows and one is never wrong. This is why wymyn like Donna, and my former coworker at Bell Northern Research, have been in my experience somewhat more loyal to good praxis including structured programming.
Moreover, I have found consistently that good structure tends to rise through the levels of a data system and that conversely bad practice down deep tends to corrupt the data system upwards.
For example, the Macintosh until now was based on a somewhat cleaner Motorola chip and not the Intel 8xxx series, and the latter's early versions included a number of design "compromises", which seemed to have infected Windows to this day.
Investigations into 2000 and 2005 voting fraud are discovering in general an "unstructured" situation of security holes and backdoors, and undocumented features because lack of true accountability tends to be reflected in code.
Lip service has been paid to Dijkstra. I don't count Ed Yourdon's praise as such, but for many programmers, there is a schizophrenia in which they admire the integrity of a man like Dijkstra but have no way on earth of demanding that their employers give them time to prove their software correct...either by formal methods or else by simple, clear, "informal" English such as would emerge in a truly fair and open structured walkthrough, with no managers present to renarrate the English as "gibberish" because the managers themselves are so corrupted by management-speak that mathematical clarity is (as GWB said of Al Gore's ideas on Social Security, and its math) to them "fuzzy".
The structured methods, and later good OO praxis have in my experience died a messy death in any organization that overvalues obedience and control, and my sources on Diebold (the maker of election software despite the fact that its CEO was a rabid Bush supporter) describe its MIS culture as one of ... fear and intimidation, the same sort that obtained in my experience at Montgomery Ward.
Management theorists since Frederick Taylor have been as is Ed Yourdon anti-Taylor. They do not believe, as regards MIS, that we don't need the structured or OO methods as long as the "mere coders" can be persuaded to forgo a family life and work free overtime "pondering weak and weary over many a midnight dreary" unstructured and obfuscated code.
Who are understandably anxious about their very nature as men (are we not men?) owing to postfeminism and the disempowering antisexuality of the corporate monastery and who seek to restore it by jettisoning the girlieman structured methods, usually at a key point...say by adding an undocumented "macro" capability to software under pressure at Diebold, which will allow operatives to change the landslide for Barack Obama in 2008 to one for Jeb Bush at the last minute.
Management literature, of which Ed Yourdon is a master, cannot even outside of Taylorism really "bite the hand that feeds it" by analyzing "the decline and fall of the American programmer" less as a failure of personal discipline than as a mechanism where management needs, almost in spite of it, to make actual skill as physically alienated as possible...if necessary, by outsourcing it, in a physical alienation that reflects the underlying economic dynamic that Marx described.
Oops. Stink bomb. I can't write management literature because at critical moments I think about Marx, and his bottom line: that no, society isn't "rational" today and that no, managers aren't wealth maximizers, seeking Profit by maintaining excellent "employee relations" who feel that the Company is their loving Parent. Even though this might produce maximum profits for the firm, as it seems to have done for companies like Southwest Airlines, it is always the exception, never the rule, always the bridesmaid and never the bride.
Sure, I'm on record as saying it makes sense to outsource to the coral strand. Hell, I even outsourced myself to beautiful downtown Shenzen in Red China.
But even in an outsourcing case, it seems to me that at the point where the remote Chinese or Indian programmer becomes actually familiar with code and invested in it (losing his initial alienation), the firm then seeks to re-alienate, by pushing into the hinterlands of China, the high country of Nepal or the mountains of the moon for "cheaper" labor. Cost pressures can't completely explain the dynamic, because it's net cheaper to stay in Bangalore if the homeys finally understand your business.
No, the underlying explanation has to take alienation, the necessity that your employees be to a certain extent Dazed and Confused, and their code reflects this, as a primitive. Marx wasn't completely right: the process isn't explained solely by profit maximization. There's a deeper dialectic in which what has to be avoided is Hegel's mutual recognition (CEOs, in my experience, can't make good eye contact and could use lessons in charm from their lowliest Customer Service Reps).
That's my idea. We think we should be rational and structured, but there is (despite good CEO intentions) a *telos* in organizations, independent of methodological individualism, that necessarily changes them from communities of rational mutual recognizers to anti-communities of people who hide in cubicles and write bad code.
Take Google (please). It has been perhaps corrupted by its adventure in China, and by allowing the Chinese government to use its search engine technology to PREVENT openness and transparency. At a key point Google sacrificed the whole deal to short term profit. On schedule its organizational culture seems to be changing to standard-issue American Fun Corporation, where people don't speak the truth, worry about their quarterly performance reviews, and stab each other in the back.
Hegel viewed history as necessary progress to a community something like the European Union in which we gain maximal freedom and recognition by being nice people, being painfully Politically Correct, and never putting people to death for heinous crimes. Alexander Kojeve literally believed this and after reading and lecturing on Hegel took part in the EU's founding as a direct result.
But Theodore Adorno may have discovered a darker *telos* in which we regress, in a process he called "psychoanalysis in reverse" to GREATER isolation (programmers sobbing in cubicles while we Call Security), GREATER levels of sexual and ethnic fear (cubicles decorated with racist "free speech") and GREATER abilities to simply ignore what is being done in our name (a BBC report this morning confirms that as of today, 90% of the Iraqi children we liberated are clinically malnourished). Which are positively correlated in my experience to the disappearance of the structured walkthrough, because Weinberg's structured walker throughers are above all sexually and emotionally secured psychoanalyzed people who don't get easily flustered.
[Not "financially" secure, perhaps. But it helps.]
Dijkstra himself, I believe, did not himself realize what he was calling for when he called for mathematical provability because ultimately mathematical provability of a data system that affects peoples lives (most data systems, that is) means you can use clear natural language, as Euclid did, to explain it in a Town Meeting to grown-ups like my grandfather, who only completed 8th grade but was no damned fool.