How to Configure Asynchronous COM+ Loosely Coupled Events

You are a braver man than I am, Dan. I took one look at com multiprocessing above the level of DoEvents and decided not to use threads in COM at all. But in VB.Net beta, I could see that a clear semaphore model was in use and supports clear notions of atomic state transitions, etc.

There are of course gotchas in VB.Net threading, the major one being that the atomicity isn't at the level of the VB.Net statement but at the level of the CLR instruction. But overall it's much clearer than in COM+.

In dealing with .Net threads, I theorized early on that as far as I was concerned there are four distinct types of classes:

(1) Stateless classes

(2) Stateful classes fully threadable, both in the sense that ONE INSTANCE of the stateful class could run procedure code in competing simultaneous threads and in the sense that MULTIPLE INSTANCES of the stateful class could run in competing simultaneous threads.

(3) Stateful classes serially threadable, in the sense that the second condition in (2) is met, and these classes may run multiple instances in competing threads.

(4) Everything else, for if a class has state (in the sense of global variables that occupy storage at run time) and cannot be proven stateful(2) or stateful(3) it is assumed "fully not threadable".

Making a class type (1) is easy. Just don't have state. Of course, this severely limits what the class can do. Classes forced (by managerial ukase and preference for popular buzzwords such as "stateless") into statelessness wind up in my experience generating secret "scapegoat" classes that in ugly fashion hold state, and pass it just in time (just in time, that is, to create confusion) to the "stateless" class as a series of ugly and confusing parameters.

Making a class type (2) is almost as easy and nearly all classes in a straightforward environment can be type (2), such that multiple instances may simultaneously run: just make sure all Shared variables are read and changed through a lock, whether an Interlocked method in the case of value objects or a Synclock in the case of reference objects.

Most of the classes in Build Your Own .Net Language and Compiler are type (3). They mostly have a single Shared variable, _INTsequence, which keeps track of the number of instances created in the current process and which is atomically incremented, and used to form the default instance name as .

About the only problem is that in forming the default instance name, I had to remember to both READ and CHANGE the variable in one call to Interlocked.Increment.

This psychologically violates "structured" programming in a minor key, for ideally you would do one thing at a time, but my early assembler experience with a strange set of IO instructions came to my rescue: my first CPU featured an instruction that would both print a line and read a card, and since this was cool I would structure assembler code loops around its use, thereby replacing dull and literally noisy code with fast and quieter code: literally, ka-chung, ka-clank was replaced by a more atomic ta-pocketa as the results for one punched card were printed out while the next was read.

This, and modern .Net threads, are what I call the fwizz factor in computing!

Type (2) is harder to develop though not, as far as I can tell, as hard as using threads in COM. Here my experience is that you need to code a dispatch() method which knows the names of all Public procedures, and dispatches to private implementations of the Public functionality inside a lock.

For simple type (2) objects, the complete state can be wrapped into a Class (that to keep things simple violates the rule that its Public members should not be variables). The dispatcher can then Synclock the class.

But there are cases (such as one I am currently working on at my day job) where locking the complete state is not the answer, at all, and creates *de facto* nonthreading in the sense that instances simply form a one-dimensional queue, and execute the one after the other as the synclocks are released...meaning you did all that code for nothing!

Here, my current praxis is not to throw up my hands and get random. It is to instead "minimally partition" the original state into a small number of separate "states", and, correspondingly, document the rules as to what Public procedures can multithread.

The process is "documentation driven" in the sense that any praxis has to conform either to the taxonomy described above, or else to overriding rules in the object reference manual. This is because in my experience, just as soon as you start using multiple threads without being to describe in clear and natural languages what it is you are doing, you are in a heap of trouble.

Some programmers delight in code that seems to work, code that demands their employment, code which cannot be spoken of except gnomically, and code that results in perverted vigils as the team gapes dully at screens at 3:00 AM. I do not.

My method is used in qbGUI as published last year, but I am not completely happy with it; it is only that I am less happy with the various alternatives. The graphical user interface of the quickBasicEngine does support availability of the form while code is running and a Stop button using Type(2) threading but the Stop button does not work crisply.

Also, a user (Randy Howard, apparently of Austin TX) encountered an error possibly related to this feature on a high-end multiple CPU system. Unfortunately, this individual's conduct on the Net, and the way he reported the error, means that I cannot interact with him collegialy to find out what's going on in his system.

Nonetheless I am working on the Stop button feature in the next (and much delayed) edition of the compiler and will try to get access to a high end multiple processor system.

I am tempted to add a fork statement to my optional set of Quickbasic extensions thereby giving anybody who's damnfool enough to use my stuff to code real code the ability to use .Net threads through "my" compiler and interpreter. But as I remark in my book, the programmer must realize that at times objects take on a life of their own, and, like Frankenstein, demand to exist.

"I was working in lab late one night, when my eyes beheld a weary sight: my monster from the slab began to rise, and to my surprise he did the Mash, he did the Monster Mash."

You get the picture: the mad scientist is possessed by objects outside himself and must reflect on the fate of Dr. Frankenstein, or Rotwang in Fritz Lang's Metropolis: being overcome by an explicit reification of Objective Spirit. Or something like that.

Multiprocessing and multithreading seems to bring out the worst in people owing to the difficulty of its bugs, and the attractions of the apparent something for nothing, the apparent fool's gold, the ectomorphic Cold Fusion: something (computing time) for nothing (ha! to think one is getting something for nothing, after hours of work, means that you have low self esteem, Dr. Rotwang!)

Multiple threads create mental suffering of a high order, and in programmers (mostly male) socialized not to feel their own or other's pain, unlike Clinton, it creates authoritarian pathology.

[Down, Ygor, down!]

In the 1960s, highly placed Big Science managers at MIT's Multics project promised Mister Bigs that Multics would use multiprocessing to get blindingly fast answers. But so many issues were unresearched that the problems caused severe delays in the Multics project (although ultimately it was delivered). The response, by some folks at Princeton and Bell Labs, was the very name of unix, which was based on uniprocessing apart from its simple fork primitive and represented a strategic retreat from the fox to the hedgehog's way.

As far as I can tell from the record, the history of thought about multiprocessing divides into Before Dijkstra and After Dijkstra, although Dijkstra had no truck with unix per se. It was only when Dijkstra directed his formidable intellect and dedication to truth at the problem was any progress made. My understanding is that he may have been the first to capture the essence of the problem and its solution in the idea of a completely atomic event.

It was a philosophical rather than technical solution in that it is creative ontology to declare that what we need is a "semaphore" rather than bemoan or workaround its absence.

In my own experience, I have usually seen philosophical computing ontology rejected. Instead of stepping back, and seeing if we can develop that which we all know is missing, "teams" are instead told by managers (in the ugly pose of the high school football coach) to persist in developing the solution as initially specified by noncoders, and this pathology is especially prevelant in multithread efforts.

.Net threads are in this sense post-Dijkstra because they give you a simple answer: Interlocked and Synclock.

Which isn't to say that you cannot get into trouble with these constructs, of course. By default qbGUI doesn't expose the Stop button and its own multithread capability has not been integrated with its constituent objects, which are all type (3).

Finally, a comment about type (4). It's an epistemological category because the standard declares that if you cannot PROVE your object to be of types 1, 2, or 3, then it IS type 4. Better safe, than sorry.

[Are we not men? Yes, Ygor.]

Great stuff, Edward. Can you provide a concrete example of your (1), (2), and (3) types?

In my day-to-day working on high volume, highly scalable, transaction-oriented, web-based systems (with much of my code residing in COM+), I deal mainly with plain stateless classes (your type (1)). In this situation, the "objects" themselves are much less object oriented in the traditional sense. They tend to be much more structured in nature. This tends to happen because cross-process and cross-machine marshalling becomes a big factor when you have multiple tiers running statelessly on farms of web and application servers. Class interfaces must be designed to optimize marshalling and reducing the number of calls required to perform a given operation. So the most outward-facing methods in the business tier tend to be very use case oriented, with things becoming more generalized, abstract, and granular as you go down layer by layer.

[quote="Edward Nilges"]Some programmers delight in code that seems to work, code that demands their employment, code which cannot be spoken of except gnomically, and code that results in perverted vigils as the team gapes dully at screens at 3:00 AM. I do not.[/quote]

This reminds me of my favorite Dilbert cartoon (someone needs to write a Dilbert search engine so I can find the cartoons I want when I need them).

Wally and Dilbert are in a conference room with another guy we don't know. Wally or Dilbert says to this guy "What's your secret? You come and go as you please, take long lunches, and walk around like you own the place. What gives?"

The guy replies, "Fifteen years ago I wrote 15,000 lines of spaghetti that runs the entire payroll system."

Wally and Dilbert genuflect: "The Holy Grail! We're not worthy!"

The guy winks at them and says "You boys will see a little extra in your checks this month."

Dan

Caution, the code is extempore. I discuss two thread models, the two most useful: stateless objects and serially threadable objects with a shared variable managed through interlocking. In a subsequent post IO shall provide code examples of objects that have state yet can fully thread.

Public Class stateless

' A "stateless" class consists of no global data (other than
' constants and type declarations). It can return constant
' information and evaluate functions as black boxes based on
' parameters only. It is free to declare local variables other
' than static variables.
'
' Stateless classes have Shared methods exclusively and you don't
' need to (and stylistically should not) declare an instance using
' a new() constructor when using a stateless class. Just use the
' class name and think of the stateless class as ectomorphic or
' transparent as compared with all other classes in this taxonomy.
'
' C# sharpies call these classes Static.
'
' The other classes must exist "concretely" in the world of things,
' but stateless classes are the pure or fixed Idea of one or more
' pure (reinen) procedures.
'
' As such, stateless classes are an excellent choice for tools 
' libraries written in the highest form of "functional" binding.
' They are also very efficient. But like angels in theology they
' lack the ability to interact with the world of men.

     Public Shared ReadOnly Property ClassName()
         Get
             Return "stateless"
         End Get
     End Property

     Public Shared Function returnLog(ByVal dblN As Double)
            As Double
          Return Math.Log(dblN)
     End Function

End Class

Public Class seriallyThreadable
 
' A "serially threadable" class can run simultaneously in more
' than one competing thread, but only as distinct object
' instance for each thread. The only variables shared by this
' type of class are Shared variables which live in an area 
' associated with the class and not with the object instance.
'
' Serially threadable classes, or classes that need to be seen as
' such, are probably the most common type of class. Multiple
' instances can run but must work on separate data and communicate
' only via their caller.
'
' If our image of a stateless class is that of an immaterial Angel
' which does not exist in the world of men, a stateless class may
' be thought of as a gentleman able to keep his paws off other
' people's data, but who insists on a right of property.

     ' --- Here is the object instance sequence number. I have
     ' --- prefixed it with an underscore as a notational
     ' --- convention. It is Private, considered as a name, for
     ' --- each object refers, using its own Private ID, to
     ' --- ONE location in Shared class data.
     Private Shared _INTsequence As Integer

     ' --- Unlike a stateless class, this object has state, and
     ' --- notice how I slide from saying, stateless CLASS to
     ' --- "stateful" or "state-ridden" OBJECT. A CLASS is pure
     ' --- (reinen) code. An object is a mortal being with a 
     ' --- soul of code, and its "sins" and other dross in state.
     Private BOOusable As Boolean
     Private STRname As String
     Private COLcollection As Collection

     Public Sub new()
          ' A REAL new() constructor should, in my opinion, take
          ' responsibility for its own success. When it creates a
          ' reference object, it should check for success. On
          ' failure, it needs to recognize that it is "deformed,
          ' unfinished, scarce half made up" and mark itself
          ' Not Usable.
          STRname = CType(Interlocked.Increment(_INTsequence), _
                          String)
          Try
              COLcollection = New Collection
          Catch
              ' Dammit, I needed that collection. Although I
              ' (speaking as the Object Instance) need to
              ' enter the world, I know I am no good. Therefore
              ' I shall exit before marking myself Usable
              '
              ' (No, I do not want to "set myself to Nothing").
              ' With centralized garbage collection, this isn't
              ' possible. As a damaged Object, I nonetheless have
              ' an ethical responsibility to .Net to Act Right
              ' until they come and take me away!
              Return 
          End If 
          BOOusable = True    
     End Sub

     Public Function tellmeSweetieWhatsMyName() As String    
         If Not BOOusable Then
              ' If the constructor failed, or a serious internal
              ' error occured elsewhere, I need here to record
              ' an error and return a suitable default. Here in
              ' this example I shall just return a simple default.
              '
              ' In this particular example, I won't say my name
              ' if I have been disposed, if my constructor failed,
              ' or somewhere else in a real solution, a serious
              ' error occured. In practice, one might make an
              ' exception for a simple Name, and other methods
              ' that do not need, for example, Heap data that may
              ' not exist.
              Return ""
         End If
         Return strName
     End Function

     Public Function dispose() As Boolean
         ' Here, I need to get rid of the collection, and mark
         ' myself unusable. Getting rid of reference objects  
         ' created by the object instance that are in state is
         ' VERY important since otherwise they clutter the .Net
         ' heap with dead soldiers with nonzero reference counts,
         ' that the garbage collector does not see, to cart away,
         ' from off the field of glory.
         '
         ' But I also must mark meself Not Usable since I don't
         ' have the needed reference object.  
         colCollection = Nothing
         BOOusable = False
    End Function

End Class

Note, in the above, that I talk about the moral responsibilities of objects. This is because computing has, in my view, a forensic dimension, in which the "mere" coder remains responsible for his objects, which as entities are his proxies.

In non-object code, the "virtual" constructs that appear are epiphenonemnal as regards the lines of code and only a first-rate computing imagination can fully retain a vision of the entities so supported. But a first-rate, or overactive, or merely caffeinated computing imagination is thereby like to generate many extra functions and subroutines for managing the virtual object that textually are only weakly linked.

The notion of object construction and garbage collection, by clarifying object lifetime (its mortality along with its "soul" of immortal code), makes the object "like" a person with rights and responsibilities to a community which is proxy for the moral community of programmers, users, and managers.

For this reason, it makes no sense to allow a constructor to finish in situations where you can detect failure, such as the failure to construct a needed reference object. But, the ONLY entity that you can terminate is the object itself; the glory days of yesteryear, where a single mainframe program could bring down the entire computer in a Wagnerian finale, are gone.

In fact, if the object contains more than one reference object, and in the constructor some objects are created successfully but then an object is not, and the constructor returns without marking the object instance Usable, the object itself should dispose() itself, and the dispose() routine needs to test each reference object that it either Sets to Nothing or disposes for Nothinghood.

Otherwise, a "leakage" of the successfully created objects will occur such that we have to trust the object user to execute dispose() on the Unusable object.

The only foolproof solution is to dispose() in any case of Unusability, taking care in dispose() to check that you are not disposing Nothing in the form of a reference object in state that hasn't been created.

I believe that there is a solution to these considerations in the form of a new programming language with a "rich" object (as opposed to the thin Object of .Net, a thing with only GetType and no properties).

The RichObject, the top level object in my Language, would have properties including Usable, Name and whether it can be instantiated in the first place. Furthermore in my Language, one could compare disimilar objects for their list and types of properties would be "first class" properties of the object itself.

But all that is More Fun Work.

A further note. It is easy to violate thread taxonomy when your code accesses the functions of MS-DOS.

Suppose two competing threads read a file. If it's the same file, they need separate internal locks (using Interlocked) on that file.

Where do you put the lock?

It can't be an MS-DOS facility, such as a file or a Registry entry, for this creates infinite regress: you have to lock, access to the lock.

It has to be in Shared variables associated with the class, but even here there is, I think, a danger. If a Windows PROCESS invokes .Net to create multiple THREADS, they all see the Shared lock which controls access to MS-DOS.

But two PROCESSES might run the same class!

I'm pretty certain this is a real danger and I need to check it out. I will report on my results in the next comment along with presenting a fully threadable class with state as simple extempore code.

Hi Edward,

Thanks for taking the time to write and post that code. I've been having fun studying it and your comments.

I had suspected from your earlier post that you and I had two different ideas in mind while using the same word: "stateless." I see now that you were thinking of statelessness as meaning "instanceless," aka "static." My concept of a stateless comes more from a COM+ context: objects are instantiated and participate in DTC transactions, but do not retain any state between method calls because COM+ in the background is actually destroying the objects between calls. In this sense there is a conceptual similarity to the notion of "static." (And do they really think so poorly of VB programmers that they have to dumb down "static" to "shared"?)

Given this disconnect I was having trouble from your original post in connecting the stateless concept to threading issues. But now, having read this latest post, I see that connection.

Can you tell me more about _INTsequence? Is this just something you're using as an example to demonstrate the threading issues, or is this something you do in practice for production code? If it is standard practice for you, what is the utility of _INTsequence?

[quote="Edward Nilges"]Note, in the above, that I talk about the moral responsibilities of objects. This is because computing has, in my view, a forensic dimension, in which the "mere" coder remains responsible for his objects, which as entities are his proxies.[/quote]

I like very much this idea of our code being our proxies in the world. It adds a whole new dimension to the concept of personal and professional responsibility for our work.

Regarding your "usable" varaible: I notice you don't expose this as a property in the public interface. How then would client code instantiating an object from this class know whether or not the object is usable--that is, whether the constructor was successful? And would every public method, like tellmeSweetieWhatsMyName(), check BOOusable before doing anything? And would it be better for tellmeSweetieWhatsMyName() to raise an exception in the case that BOOusable is False?

I'm also curious about your use of an all-caps Hungarian convention. I take it you're not in agreement with the movement within the .NET world to eradicate Hungarian? And why all caps?

[quote="Edward Nilges"]In non-object code, the "virtual" constructs that appear are epiphenonemnal as regards the lines of code and only a first-rate computing imagination can fully retain a vision of the entities so supported. But a first-rate, or overactive, or merely caffeinated computing imagination is thereby like to generate many extra functions and subroutines for managing the virtual object that textually are only weakly linked.[/quote]

Edward, you gotta help us mere mortals (the ones without philosophy degrees anyway) with this one. :-)

I'm looking forward to the promised type (3) code example.

Dan

"Stateless" means that there are no variables preserved between method calls so it means, I think, the same thing a static. Because of VB's sloppiness, you can "instantiate" a class with no state in this sense, whereas in cleaner languages you can't, and static is a reminder that you can't.

Being able to instantiate a class with no "state" has no application, of course, except as a zero operation to see if it can be done while testing the compiler.

The actual objects in Build Your Own .Net Language and Compiler have in fact a read-only Usable property and a separate mkUnusable() method. The Usable property can be examined to see whether it is OK to use the object, but it can't be written because user code should not tell an Unusable object from the outside, "you, Unusable object, are in fact Usable". There is no application for this and it violates common sense security.

But the reverse operation is allowed through mkUnusable() if a user object senses abnormal behavior. This might be a security hole in that a bad user object can mark all objects it sees as Unusable, and disable a process thereby. I decided to implement it in the objects in Build Your Own because it biases the system to not proceeding in possibly erroneous situations, of which a "bad user object" is one: at worst, it stops the other objects from proceeding.

The Hungarian notation is explained in Build Your Own. Module-level "state" variables that are not Structure or Class members have upper case Hungarian prefixes, while local and Static variables have lower case prefixes. This visually reveals the lifetime of each to the reader of the code.

I can see NO good reason AT ALL for this postmodern rejection of Hungarian notation. No one programmer is at once so verbally facile and knowledgeable about his possible readership to choose the "most meaningful" non-Hungarian name: indeed, the very idea of "most meaningful" is at once introspective in a malign sense: an appeal to the mob: ethnocentric: and statistically, meaningless.

The all-important selection of type is therefore a valuable addendum to the "hopefully, most meaningful" name. This is because the selection of type is an important forensic decision (in the sense of a decision with moral and business consequences) that needs to be relatively immoveable, and performed when the variable is created.

A great programmer knows or discovers the range of values a variable can have: a lousy programmer randomly changes data type, while debugging.

As such the variable type may be compared to human gender, which identifies people on documents because of its fixity, and in most cases its determination at birth. Which isn't to say, of course, that gender, or variable type cannot change.

I conclude that the abandonment of Hungarian notation was performed as an Oedipal gesture of revolt.

Finally, what I meant in the last comment was that in C, a complex data structure is often best implemented with auxiliary routines to check its validity and print it out, but these routines are extra work to write and maintain, and can cause bugs when not packaged with the original data structure.

Whereas in OO, the "complex data structure" CONTAINS methods like inspect() and toString(), bound to its core "with hoops of steel".

I shall endeavor to be clear in the future.

The Shared variable _INTsequence appears throughout the code of Build Your Own as a simple sequence number identifying the order of appearance of the object within the Windows process.

It starts with an underscore to show it is Shared: its Hungarian prefix is upper case because its name scope is object-wide (I apologize for using "module level" in the previous comment: it were a brain fart: it is VB-6 terminology: VB.Net files consist of classes).

Since in practice multiple "coding standards" are the default, and because a Structuralist consistency is more important than any one coding standard I document my standards in Build Your Own without any implication they are Great. (Structuralist in the literal sense: a program is best understood not as a set of fixed referents but as a structure of identity and difference: understanding a program as the former creates unmaintainable code).

Whereas when altering another programmer's code, I attempt to determine his coding standards and follow them exactly. This is because over and above ethnocentric wars, the consistency of the code is very important.

But in following my own standards I never deviate in an almost autistic sense. This is because if they are good standards then they should be followed (this is "obvious, profound, stupid, or all three").

I don't want to pollute these fine posts with RTFM-kind of questions, or some "below the level" comments. I just have to say that people (hopefully developers) should have (and read) discussions like these more often. Objects are objects, no matter what. They should all follow the same rules (yes, rules instead of standards) across languages and technologies, if we are hoping for a better “programming world”. A silly VB6 programmer like my self some times lingers over these kind of topics, and is refreshing to know that all-programming-languages/techniques literates some times ask the same questions that I do. The big difference is they actually conclude to something useful for us all.

A while ago I was talking to a colleague about useless conversations. "You always know when you had one of those", she said "You walk away feeling a little bit dumber". I came here looking for some guidance on MSMQ and atomic transactions, and I walk away feeling a little bit smarter. So, Daniel and Edward, in the name of us “grasshoppers”, my deepest thanks to you guys, for sharing your knowledge and experiences.