c++ Windows design framework (proposals and collaboration): beginnings

[galathaea]

After much delay, procrastination, and downright laziness, I have decided to bite the bullet and post a project. Its a big project, but I think it is one that many might be interested in. Here is the background.

I have come to dislike MFC to the marrow of my core. I think it is an absolutely hideous framework for manipulating the Win32 API, and is hardly by any means object-oriented and not at all design oriented. I couldn't begin to list all of my reasons here in this first post, but those familiar with some of my posts on the subject will already have a good background to my reasons, and we can discuss particular points as the thread evolves.

Here is what I would like to do: Over the course of this thread, I would like very much to explore various third-party libraries and frameworks, discuss important idioms, and generally outline the components that can be used to build a truly professional framework for software design in c++. Along the way, we can venture into the territories of other languages that may have features that c++ can emulate for more advanced design structure (SmallTalk, Lisp, and c# come to mind). I want the opinions of actual professionals in the field as to their solutions to various design problems, so we can organize a general collection of important elements a true design framework solution must have.

I have worked on this for some time on my own, but understand that I do not have the experience to fully see certain design problems I have yet to encounter, and it is also obvious that this is not a one person endeavor. We have all had to find ingenious solutions to design problems, and many of us already have quite a store. My suggestion here is that we share and debate.

I have many initial proposals to help guide the early discussions, and will try to download to this thread many of my notes on the various issues over the next weeks as I get time. Here is the basic structure that I believe a design oriented framework must encompass in the windows environment.

First, it is OK to be Windows OS specific, but generalize as well in the hierarchy. One of the problems with many design books is that they stay away from OS specifics in an attempt to show the strength of general c++. But the goal here is to describe a framework for Windows programming. There is much that can be done with general assumptions about an OS (see Qt for example), but more powerful work needs to focus on the actual environment.
The OS environment needs to be encapsulated properly. The objects in the environment are things like Hard Drives, Printers, Displays, etc. The framework should generically encapsulate these objects and provide standard interfaces to control them and send messages/data to and from them.
The concept of an execution segment needs to be properly generalized to the multithreaded, multimodule, multiprocess environment that they exist in ( a generalized function concept).
The concept of threads needs to be deeply encapsulated to describe the true execution/branching/messaging that is required for modern applications.
These thread segments should be handled like functors for constructionof the execution tree.
Branch forms need to be generalized from the logical variable conditional comparisons (direct calls), function map index calls (pointer calls) thread creation, and higher order derivations such as that found in object polymorphism and factory returns.
The primitive thread control mechanismssuspend/resume, terminate/begin) and their more developed consequences (swaps, branches, etc.) should be encapsulated as part of the framework.
Messaging types such as the numerous elaborations of shared memory (stack parameter passing, heap, global, interprocess) and the OS supplied messaging patterns and methodologies (queues, pipes, etc.) are necessities to be properly designed pieces of a solid framework.
Interthread execution synchronization needs to fully represent modern synchronization theory in the available design idioms.
Objects should be easily inserted into object control paradigms. This means generic use of factories and junkyards should be available, and that object passing can be encapsulated in various smart pointers.
All of these should be policy based design implementations to allow for flexibility amongst all developers on a team.
The smart pointers should have variations to include access to interprocess memories.
"Message loops" (Invokers and/or Dispatchers) should transparently encapsulate their messaging relationships in a dynamic method that involves registration idioms, as well as allowing compile time binding of these connections.
Window/control handling should follow a general skinning paradigm, encapsulating user-interface items and connecting up the graphical input message dispatching without requiring intermediate registration and allowing dynamic graphical representation.

There are all sorts of things that have been missed in this initial list (socket dispatching, thread/fiber pooling, ...) but this is at least something to focus on. I understand that everything may sound overwhelming at first, but the point of my posting this is that there is a HUGE resource pool out there that have amazing intellects and already have accomplished much of this independently in their own work. Its definitely an accomplishable task, since much of the work is organizing already made libraries and possibly working on specific coding subprojects. As the project grows we can develop separate threads to work on subprojects in an organized manner.

I understand that the general pattern of interest may show a common ebb and flow of interest. I am committed to developing the issues to the extent of my ability, but hope that there are many out there who wish to contribute. I will try to regularly organize and summarize the work to the best of my ability and hope that others wish to as well.

So, perhaps to start, it may be best to take the first series of posts to encompass general suggestions, organizational proposals, third-party library referrences, and other general contributions. I will be preparing follow-up posts to elaborate particular design patterns I have implemented. After that, even though I am sure I can not in any way control the flow of this thread on such an interesting and diverging topic, if things lead that way I will begin posting proposals to generalized function idioms that fit the Windows thread/process/module model and work on design frameworks on their invokation and dispatch, as this seems to me to be the core or engine of a good framework (and happens to be the area I have personally most developed).

For now, I definitely consider good starting places to include the Loki library of Andrei Alexandrescu and the many brilliant libraries at boost.org. I desire as many other library suggestions and framework analyses as there is experience with out there (OWL, etc.).

I hope to hear from many of you.

[TheCPUWizard]

I am definately interested in persuing this topic. Our firm has developed a number of libraries (as well as using 3rd party libraries) to accomplish a number of these goals.

Our primary limiting factors in producing a comperhensive general use framework have been the following:

1) Man Power. :(
2) Need to ship (paying) projects. ;)
3) Designing with our own targets in mind. :rolleyes:

By exchanging ideas with other developers we hope to gain some additional insight on how others envision the architecture. We also hope to provide some concrete samples of our designs.

On a more specific note, the .NET / CLR framework does support a larger number of these features and has a reasonably consistent approach. Although the CLR is designed to be an Open standard, most tools are strictly for use in an MS environment (the number of non-MS and even non-Windows tools is increating though...)

[galathaea]

Here is a furthur expansion on what is intended here. I really believe that there is already some major advances available in the various libraries out in c++ land. As well, there are many specific solutions to focused problems scattered all over the internet (many solutions can be found here at CodeGuru). Some good summaries of past libraries can be found at the c++ faqs (www.trumphurst.com/cpplibs/cpplibs.phtml) and this list (www.desy.de/user/projects/C++/Projects.html). Also, on the same site as the latter list is this useful comparison (www.desy.de/user/projects/C++/Libraries.html), though I must admit that the site is quite old. I think it is important to also post links in this thread (and other supporting ones, should they be started) on particular finds relevant to whatever is currently being discussed, so that we have a document path for new readers of the thread and have available referrences for study. This includes links to other threads here in these forums and other related threads in newsgroups like comp.lang.c++.moderated.

What I desire from a framework solution is threefold:

It should be open-source. Not that anyone using the framework should need to see the source to use it, but purely because it is the only way we can honestly debate the merits (and potential pitfalls) of a particular solution. This excludes several popular frameworks from inclusion into such a project, but it does not exclude debate on the paradigms of these frameworks. It only implies that without coded implementations, we will have to work on the code ourselves if a given solution is deemed best.
It should be flexible. I do not believe that any professional desires a framework that forces idioms on them when several choices have applications under different environments. This is why I desire policy-type solutions (or other forms of polymorphic design patterns). For me, flexibility also implies a structured, likely tiered, approach so that the framework has potential portability for the most "pieces" that do not rely on OS specific methods. This does not mean that I desire an OS independent solution. The framework should still encapsulate the powerful features of the OS (which is why this should start out as a purely Windows solution).
It should be powerful. I find that much time is spent in development re-implementing common design patterns instead of focusing on the application specifics. This is what I want to change.

Now, I usually operate in the following manner. I am usually presented problems or goals and work out solutions. Most of my learning of the more esoteric idioms of c++ has occurred when I was studying referrences to find the solution to particular problems. Even though I have learned alot from reading books and then finding problems for which the books have given me solutions, I have found that I understand an idiom or design pattern much better after I have independently found myself in need of a solution. So I think a good way to start out the discussion of a particular area of the framework design is to give lists of problems we are trying to solve by the implementation and then, once our goals are spelled out, discussing the various merits of solutions. In this way, we can begin generally and focus in on particular areas of discussion as the ideas become more specific and detailed.

Although I have many scattered ideas and various solution proposals for many of the areas presented at the start of the thread, I believe now that we need to begin with more fundamental questions first. And it seems to me that the beginning of any design framework is squarely centered on the object and pointer models to be used in the rest of the work. So perhaps the first question that needs to be pursued is

What are the best idioms to use for object/pointer models in a framework?

The general word best is key here, as we need to define what parameters we are using to compare solutions and where the answers may differ in various pieces of one's design. Answers can focus on the merits and weaknesses of such solutions as MFC's CObject, component models like those in c#, and the numerous other solutions out there. Even particular object characteristics, like those found in Volker Simonis' chameleon objects (www-ca.informatik.uni-tuebingen.de/people/simonis/papers/chameleon/value/value.html), can be debated. All of the great, smarter pointer types (master, brilliant, facet, handle, et al.) can be looked at, and the best implementations of these idioms can be debated. We can discuss the general applicability towards real world difficulties faced in application design. Here, I believe, I will really need to look to those who have many years of professional experience, as, in many ways, even though I can debate the semantic benefits of various solutions, I am still in many regards fresh in the field, and may not yet see some potentialities.

I am glad to see your post, TheCPUWizard. I hope this is the start of a vigorous project. If (when) we get a small initial group interested, we can petition Brad to put us in our own little area, where we can comfortably crunch away at the various pieces of this large project. For now, I will continue to post here various details and concerns in getting the project started as time permits and am committed to posting all of my study in this area. With TheCPUWizard on board, it already looks like there is much material to discuss!

[TheCPUWizard]

OPEN SOURCE - I am in complete agreement that open source has some major benifits as a model. It also has some issues. As (a selfish) example, our current disorganized framework is the result of man years of development. It is only because of this framework with all of its limitations) that we are able to rapidly produce high quality products for our clients. Our typical license agreement, gives our clients full source code to the project specific code, but there is an additional (very large) charge for the source to our existing libraries, especially if the client is a development house themselves.

Perhaps a compromise policy may have some benefits. Concepts, Documents, Headers, and Models would all be Open Source. Implementations can be kept as either Proprietary or Open Source. Therefore software developers could create libraries which were compatible with the (to be developed) framework standard, but would still have licensing control over their own products.

The scheme outlined above is basically that which is used for Active-X component development. This has method has been proven to (generally) create components that interact together and also allow professional developers to remain in business.

Please note that I am not ruling out participation in an Open Source project. My concerns include issues that would arise out of USE of this code if it were to be modeled along the lines of GPL or LGPL.

[TheCPUWizard]

OBJECT / POINTER MODELS - This is a very good place to start. If the basic object interface and methods for instanciation of and communication with these objects is exceptionally well designed, the framework as a whole has a decent chance. An error at this stage could prove to be fatal to the entire project.

Although an overview of my approach to design may be apparent in a number of the posts elsewhere on CodeGuru (and other forums), I would like to state some key points here:

1) Develop your code so the computer can catch errors as early as possible. Compile time is best, runtime checks are better than no-time checks. As an example, if objects of your class are not to be copied regularly, declare a private copy constructor. If there are specific circumstances where an object may need to be copied, but you want to make sure the "computer never does it for you", implement a special method such as void Clone(source).

2) Have built in support for different build configurations. We should all be familiar with Debug vs. Release Builds. I believe this needs to be taken to another level. Remember debug builds (at least those that link against an MS debug library) can not be legally distributed. We typically support a "Trace" build that enables a number of debugging features into our objects. This is independant of Debug/Release giving a total of 4 supported configurations.

3) Documentation is CRITICAL. However if there is separation of the documentation from the code, the two will rapdily diverge (who remembers [or has the time/interest] to go back and edit 20 word documents that contain a methods interface when you add a new parameter to that method)]. Integrated solutions are the best (full cycle design tools).

My next post will contain some SPECIFIC ideas on Objects and Pointers, but first dinner.....

[TheCPUWizard]

WHAT IS AN OBJECT? This may seem like a redundant question, but lets look at it from a non-programming perspective.

An object can be considered an entity which has a specific lifetime, performs various operations, accepts input (optional), and produces results (optional). The means by which the object performs these actions can and should be transparent to users of the object (encapsulation). We all know how to use a television, but the vast majority of people do NOT have any understanding of how it works internally.

How does this reflect in programming (specifically c++)???

Lifetime - All objects have a lifettime. This can be based on scope (including global and static) or on dynamic creation/destruction. Objects may also be referenced directly or indirectly (via pointers & references). Again this should be no news to anybody, but It does bring up important issues: The destructor of a scope based object should NEVER be invoked via delete, A Dynamic object must be destructed explicitly once and only once, and an indirect object reference mush never be used after the destructor has been called. A comperhensive object model should enforce all of these items.


INPUTS/OUTPUTS/OPERATIONS - Conventional prodecural programming implements this as a series of methods (member functions). Parameters can be either inputs or outputs, return values are always outputs (when present), and the method itself can perform what ever operations are necessary. Again not rocket science.

This model has a number of limitations most of which are the result of tight coupling. In order for the object to receive input, some other object must have a reference to the object and pass the parameter in. In order to make use of the output, some other object must periodically call the method which reports the output (again this requires a reference to the object).

Better designs can be achieved using Notification/Event methodologies. In this case it is possible (although not necessary) to implement a distribution system which directs the processing of events. If every event is given a unique "name", then an entire system can be implemented (this would be and extreme case) where no two objects communicate directly. Each object would register itself to be notified when an event occured, and in turn would generate events as a result of its operations. It is theoretically possible that objects would have NO conventional methods in this architecture.

There are a number of different models that can be used based on requirements. Some of the criteria are listed below

1) Single or Multiple Event Producer
2) Single or Multiple Event Consumer
for multiple event consumers, to they occur in parallel (at least conceptually) or do they form a chain.
3) Is the event bound to the consumer, producer, neither, or both.

OBJECT EXECUTION - Many frameworks have been written based on sequential execution. Everything can be tracked through explicit calls and returns. When modeling real world events many (most?) objects require some autonomous execution capabilities. One implementation of this is a multi-threaded model.

Classic multi-threaded models have some limitations. First, at a system level, threads consume resources and switching between threads has a fairly high overhead (micro-thread, fibers, and other techniques help here). Second multi-threading may not translate well to multi-system (e.g. LAN/WAN) or multi-cpu systems. Obviously all objects to not need (nor should) support all of these methodologies, but a consistant method of implementing each of them should be part of the framework.

DEBUGABILITY Did you know that the term "bug" dates back to the 1800's, but the term "debug" was not coined until the mid 1900's? This seems to be the case with most software implementations. If a framework is going to be used on a repetive basis, an investment here will surely pay off.

One simple case in point. Have you ever had a problem with one instance of a specific class. You put a breakpoint on the offending method call, but every instance that executes this method triggers the breakpoint. I do not want to count the number of times (would require 32 bits) that I have written down the "this" pointer of the offending object, then kept running the program and checking the value of the pointer. If you have a few objects with similar pointers, it is asy to get confused. Having a "Readable" object name for each instance is one method that can make this particular situation to deal with.

SUMMARY - I hope this starts to give (my opinion of) some tangible features to include in the core object model design. I am looking forward to feedback on these items.

David V. Corbin
President
Dynamic Concepts Development Corp.

[galathaea]

I am very glad to see we are on the same page. These are precisely the issues we need to pursue right from the beginning. Obviously, different objects have different requirements on many of these issues, so it will be important to identify various policy types, derivation types, or other design relationships for these issues. There are also some other issues, like some classes requiring exiting the c++ system (storage to disk, transmission over sockets, etc.) and need some form of typing handles in their binary data for reconstruction handlers, but I am preparing my notes and will reply to these issues better tomorrow.

However, I will discuss the concept of Open-Source right now. When I wrote that, I was thinking of the fact that there are some design pattern libraries (etc.) that already implement some of the candidate solutions, and I do not wish to use any third-party solutions that I cannot see the code for because I cannot guarantee the quality of the implementation. I always figured that a collaboration here would require us to openly post our code in those areas that libraries do not satisfactorily fill, so that we may debate the solution. I was thinking along those lines (as opposed to the standard distributed development) for two different reasons. First, that is the standard way of arriving at solutions on these boards. And probably more importantly, the solutions obtained are better reviewed and more fully debated.

This brings us to the component paradigm you mentioned. I think that might be fairest to all developers who put their time into such a project with the expectation of maintaining ownership. And we will need to get to the point of interface designs prior to coding anyways, so I see the possibility for a hybrid here. We can state from the outset that modules or components distributed on this project are explicitly not to be used for development of any saleable products without express agreement from the author (which may include some money transfer). Instead, posters have the option of debating code or presenting their components as possible solutions. In fact, one person may post code for implementing a solution where another has posted just the component. Debate will center on various visible characteristics like efficiency, stability, and the like for components, and focus as well on semantic issues for code.

Now, as for licensing, distribution, and such, I must regrettably state that I had not thought along these lines. Which is unusual for me, since I work for a startup at the moment and about everything I code is focussed on just these issues. The reason behind my initiation of this type of project, however, was because I had been slipping furthur and furthur away into pure Win32 API programming for some time and implementing various design patterns that seemed to me most suited for robust, professional applications. But my programs looked almost nothing like MFC, and when I looked into MFC code, I kept seeing more and more design problems created by the framework. Frustrated at the lack of any substantial competition (well, excluding .NET, c#, and other MS technologies -- not competition) that could fit my ideas of good design frameworks, and knowing that this was not an uncommon frustration, and as well realizing the complete project was too large for myself, I came up with the idea of posting here. I am not against setting up a general licensing plan to enforce intellectual rights, but it seems we may have to discuss the discussion mechanisms for code more deeply here at the start. I am completely open to ideas!

[TheCPUWizard]

Don't you just hate it when someone brings up "business" issues on a technical project :D

Seriously, I think your idea of a tiered participation paradigm is the best bet. Allow participation along something like the following:

1) Discussion only
2) Public code posting with free use by anyone
3) Public code posting with terms of use (including fees)
4) Component (e.g. DLL + Headers) posting (authors should make the source code available (for a fee) to interested parties

I feel that by leaving these options (or something similar) open will encourage other developers to join the efffort. When the project has matured a bit, it will be obvious which model is more desirable to the participants. This will in turn engourage developers to use that model. By doing this, we avoid the potentially deadly possibility of establishing a single method of participation that turns developers away from the project even before they really look at it.

[TheCPUWizard]

Project Communication

Right now we are using a single thread for discussion of all topics. This is going to get very unwieldy soon (probably in the next few days at this rate. I do not know if there are facilities here at CodeGuru to establish (semi-) moderated multiple discussion threads. I also think that posting code in the threads (directly) will cause a lot of grief. A revision controled repository for the code would be a great boon.

Another issues is getting more developers to be aware of this project. One idea would be for us to create a small article that could be posted here on code guru (and possibly other locations). If we can get to the point where we have some of the existing thoughts organized and are ready to propose somme solutions, I think this would be excellent for both the project itself and for the developers themselves (sort of like the old professors statement of "Publish or Perish").

[BobClarke]

Have you considered SourceForge?

"SourceForge.net is the world's largest Open Source software development web site, providing free hosting to tens of thousands of projects. The mission of SourceForge.net is to enrich the Open Source community by providing a centralized place for Open Source developers to control and manage Open Source software development. To fulfill this mission goal, we offer a variety of services to projects we host, and to the Open Source community."

[TheCPUWizard]

Bob,

First, I would like to sincerely welcome you to this discussion! ;)

As this project was originated by galathaea, I am hesitant to comment too deeply, but here are my (not so humble) opinions....

If you read through the (rather long) posts [or War&Peace], you will see that I feel a strong need for a methodology for this project other than a single threaded discussion board.

I have been involved in a number of SourceForge projects and have some (personal) comments on the site...

1) Yes it does "host tens of thousands of projects", however the vast majority of these projects are not maintained [many have never gotten past the orignal posting (no comments, no code, no docs)], The number of viable well-maintained programs is MUCH smaller. With the mass of all of the other projects, it is difficult to find a Gem among the masses.

2) Licensing. As I also mentioned, I feel the need to support multiple levels of control over posting and use of code. As the project grows, it is my hope that some vendors of commercial software will "join up". While this is definately premature, I have had experience with a number of major vendors either developing a tailored version of their library or providing me with the source (under rather strict NDA's). I am sure they would not do this if there was a REQUIREMENT that the software be distributed free of charge to anyone who wanted it (can you say Chapter 11) There are many others who would not get involved at all with that restriction

In summary, I feel there is a need to a venue with the CAPABILITIES of SourceForge, but I am not sure that SourceForge itself is the best solution.

Again, welcome to the discussion. If nothing else this project should invoke lots of great ideas as the number of active participants increase.

David V. Corbin
President
Dynamic Concepts

[TheCPUWizard]

Galathaea,

Just e-mailed you a link, wanted to get your reaction.......

David.

[TheCPUWizard]

Galathaea,

You seem to have lost interest.......:(

[galathaea]

I was forced to do some other work over the weekend and was checking out the options available as well, because I wanted to assure that the project could find a good home. I think it is clear to me now that CodeGuru (although a beautiful site for many types of coding work and questions) is likely not the best home for this type of project. TheCPUWizard has provided a quite interesting option that he has built, giving the forums necessary for just such a professional collaboration that can be expanded to handle all collaboration issues without requiring us to go to the management of a third-party site to handle our requests, and I am indebted to his commitment. He seems to have worked hard (and quickly!) to get things up, and knowing his previous work on just such a project, I suspect he had similar aims to mine prior to my posting (this is a great development!). I had suspected that there would be such an interest lying dormant out there and believe that his is a great indication of what can be expected.

I do not want to post the address of TheCPUWizard's site in my post, as I am unclear if he is ready to begin accepting posts yet. Since he has worked so hard in its development, I will let him post the address when he is ready (do it in a large size= post in this thread to make it clear to those reading this thread the new direction when you are ready :)). We can begin work on establishing various lines of licensing, etc. and get the real work begun right away!

I have some large posts ready for later on today whose main goals are centered on enumerating the many types of objects that actually occur in a professional project and work to generalize and expand on there specific needs in the application environment. This, I believe, will allow us to focus the suggestions already elaborated in this thread by TheCPUWizard and myself and will focus our attentions on the decisions and needs we can work on at this point in the development of the framework.

The only concern I have with the alternate website is the general nature of its title. Although it sounds like a fairly good professional title to entice serious programmers, I think all projects need to have a good marketable "codename" as well. For example, I was always a little turned off by the "Chicago" codename for 95 years back, but was more interested in the talk surrounding "Jaguar" for the newer Mac OSX version when I first heard it last year. Marketing is everything, so I think we can also (on a less serious thread) discuss possible codenames for the project to entice the younger, more to-the-XTreme right-out hippin' crowd. Something like (this is just out of the blue) "Singularity" or some other technical-sounding mumbo-jumbo with some off-the-wall analogy to the design philosophy held by the project. That way, alternate distributions can elaborate the name with other techie extensions like "Naked Singularity" and other such marketing distinctions. Just a thought... Also, a little more serious but nothing too important, COF might get confused with the COFF object file format, though I think we could keep that name as the official wearing-a-tie name for the project.

My sincerest gratitude to TheCPUWizard for all of his help so far on getting this project out of the gates. He has been the main contributor to date and I hope we can continue to expand the project to its inevitable end.

PS: The categories of the discussion forums at your site are right on!

[TheCPUWizard]

Galathaea,

Glad you like it. I was on a clients site most of the day. I will make the site putlic within the next 6 hours...Stay tuned everyone!

[TheCPUWizard]

New Web Site Available!

Dynamic Concepts Development Corp is proud to host an expanded forum for this topic. Please come over, share your thoughts, and spread the word!

Framework Design Project (http://dynamicconcepts.us/COF_Project)

[galathaea]

I very much am enamored with the use of a centralized (this means centralised, for our readers in the UK;) ) discussion forum and code repository for the project and will update my signature accordingly. I think it will take a couple of days to start up the various threads explaining the various directions this discussion can begin at, and I will work at building the foundational threads immediately. After the basis for such discussions are begun, I will work at getting the message out about the collaboration over various newsgroups and related message boards, to attract a wider audience to the project. I am confident that as long as such posts occur in appropriate forums, the various sites (including CodeGuru) will not be averse to such recruitments.

This is not in any way meant to discourage posts in the meantime from those eager to begin participation! Please come over to the new site and share your ideas!