Today is the day most software turned to "garbage"...

[TheCPUWizard]

It is here...8 logical processors on a reasonably priced dekstop machine.

Unfortunately it is estimated that 95% of all existing software is unable to make use of more than one processor.

[Hobson]

On topic: Generally I do not program for parallel processing because I do not need to. I have no experience in that either. But I am aware of it and when I should create / work on OS, or software performing extensive, "paralellizable" calculations, I would give it a shot.


My OT rant: It's not only that, man... I think that we need to introduce new measurement of software quality: it would be expressed in percentage of cases it works correctly. Like, you know: "This Windows XP is whole heavelny 98% good! Not that [bad word] 50% Windows 98 was!" or "This new Java Enterprise Server is just lousy 70%'er, I had problems configurating it on 2 machines, and I can't get some stuff to work on remaining ones". I think you get the idea. It's just because nowadays, when silicon is cheap, mass storage is really massive, and so on, the only things that grow are system requirements and advertisement folders for new software. Quality drops dead, "brand-new, improved, easy to configure and deploy software which makes your industry running on top levels" is usually slow, unconfigurable, undeployable, expensive. Or, is that just me?
And tell me, what is the difference in FUNCTIONALITY, ERGONOMY and USABILITY between Word '97 and Word TheNewestOne, which justifies tens to hundreds times bigger minimal requirements?

Doh, I could go with this for hours...

EDIT: feel free to discuss my points in IT Grumble thread! http://www.codeguru.com/forum/showthread.php?p=1783182#post1783182
Let's keep this thread and survey on topic :wave:

[TheCPUWizard]

On topic: Generally I do not program for parallel processing because I do not need to. I have no experience in that either. But I am aware of it and when I should create / work on OS, or software performing extensive, "paralellizable" calculations, I would give it a shot.

That is the general attitude I expect. But I have to ask, would the people who are using your programs:

a) Be perfectly happy running them on a 333MHz computer as they are on a 3.2Ghz computer

b) would they turn down an upgrade to a 25GHz computer??



My OT rant: It's not only that, man... I think that we need to introduce new measurement of software quality: it would be expressed in percentage of cases it works correctly. Like, you know: "This Windows XP is whole heavelny 98% good! Not that [bad word] 50% Windows 98 was!" or "This new Java Enterprise Server is just lousy 70%'er, I had problems configurating it on 2 machines, and I can't get some stuff to work on remaining ones". I think you get the idea. It's just because nowadays, when silicon is cheap, mass storage is really massive, and so on, the only things that grow are system requirements and advertisement folders for new software. Quality drops dead, "brand-new, improved, easy to configure and deploy software which makes your industry running on top levels" is usually slow, unconfigurable, undeployable, expensive. Or, is that just me?
And tell me, what is the difference in FUNCTIONALITY, ERGONOMY and USABILITY between Word '97 and Word TheNewestOne, which justifies tens to hundreds times bigger minimal requirements?

Doh, I could go with this for hours...

THAT woud be an excellent thread, and I have distinct opinions.....

[Hobson]

Back on topic: does anyone here know, whether Windows, or its any part, helps me to utilize CPU cores in any way, which is helpful but invisible for me? Like, when I create new thread, does OS pick a core on which it is run? When I write native Win32 app, or MFC app, anything like that, are mouse messages processed on one core, painting messages on another, and other ones somewhere else?
Does Windows (or MS libraries, like MFC and others) help me to use computational force of my CPU in a single native application without the need to use multi-cpu / multithreading APIs by myself?
I would be really greatful for any explanations, readings and info on this topic.

Cheers

[TheCPUWizard]

The Parallel Extensions library [which will be baked in to .NET 4.0] allows you to effectively use multiple execution patchs, the Library in conjunction with the OS will very nicely make use of a significant number of cores (if used properly).

[Hobson]

Thank you for your answer.

But as I understand, Parallel Extensions library will be available for .NET platform. And what about native applications?

My main point is that I think, there should be transparent, implicit support for parallel processing provided by environment used by developer. I think that it would be good thing that i.e. when I make a call to some asynchronous WinAPI function, it is automagically executed on least utilized core. Or when I use some GUI library (let it be MFC, wxWidgets, anything), it processes messages, calls handlers and paints after picking the best (least used?) CPU. I think that these are library providers who shuold think how to make their code cope well with multicore systems, not me.

You ask in your survey whether I design for parallel processing. My answer is: should I? Of course, sometimes I should, when performing paralellizable calculations etc. But in general case, should not it be components designed for parallel processing? Then, each desktop application created by me would efficiently use multiple CPUs without any effor from my side.


I hope you understand my point, I feel that I did not express myself clear enough, but I can't better...

[TheCPUWizard]

For native applications I am not aware of any Microsoft solutions at the higher level, but the information in the "slow chat" covers a bunch of lower level tools (e.g. Futures/PRomises) and their statuc in Dev10 (aka VS-2010).

There is also the Intel libraries, which I have been using in production code for a few years.

The biggest issue is that the entire philosophy of a massively mult-path system is radically different in design.

For example, using immutable objects (ie EVERYHING is done in the consturcotr and ALL methods are const) rather than locking/synchronization is a powerful tool that usually causes people heads to explode the first time they encounter id (imagine NOT being able to add or remove items from a collection!!)

[goatslayer]

My vote was, No, but I plan to.

I haven't the most prolific of software portfolios at the moment, most of the stuff I am doing is for work.

My current home project won't be designed for parallel programming, but I know its successor will be.

The biggest joke is my workplace where we have code that would run best on P4's, but we produce the servers to run it on and are now onto quad core processors. Three out of those four cores are left practically untouched.

The response from the company is that there are no plans to do anything about it.

Along with other issues, it can be like banging your head against a concrete wall to get them to move on anything progressive.

[TheCPUWizard]

....The biggest joke is my workplace where we have code that would run best on P4's, but we produce the servers to run it on and are now onto quad core processors. Three out of those four cores are left practically untouched.

The response from the company is that there are no plans to do anything about it.

Along with other issues, it can be like banging your head against a concrete wall to get them to move on anything progressive.

This scenario illustrates my point quite well. IMPO: The real problem is that the architect for the project did not design for parallization 3-6 years ago.

"Migrating" an application for single execution unit to multiple typically involes a scrap and re-write. But designing for it from the beginning is a fairly small incremental cost (relative to a scrap and re-write).

Imagive how "delighted" everyone would have bit if the first time it was fun on a Quad, it immediatelyshowed a 3x performance improvement (you never get 100% scalability due to overhead).

[answer]

I would like to learn how to code for multiple cores. I see some people have managed to optimize this way; but I cant find resources on how to do it? Do I simply create threads and the OS will put the processes on different cores? Are there specific OS constructs? I couldn't find online sources yet on how to code for multicore processes, only concepts.

The best I have found is Taking advantage of multiple CPUs for games (http://rene.f0o.com/~rene/stuff/europython2007/Taking%20advantage%20of%20multiple%20CPUs%20for%20games%20-%20simply.pdf). Which has a few concepts.

[TheCPUWizard]

Do I simply create threads and the OS will put the processes on different cores?

If only it was that simple.

The truth is that most people are taught from a very young age (3-5 years old) to think in sequential steps, to address each aspect of a "problem" individually and one after the other.

Writing parallel executing programs requires that a person breaks this lifetime habit, and develops a thought process of "how many things can I do at the same time?", "How can i address the problem so that the parts are independant?".

This is why the "thought" provide by Hobson a while back (intrinsic support) is only a very small part of the overall issue.

One great "real-world" example (which has nothing directly to do wih programming) is the TV show "Extreme Makeover - Home Edition". For those not familiar with it, it involves demolishion of a home, the building and furnishing of a home in under 7 days for a deserving family". I can attest that they actually do it in that time frame because one of the projects was in a town about 7 miles from where I lived).

They approach it by having a few hundred contracters (and often upwards of up to a thousand volunteers) all doing whatever they can, as early in the project as they can, with minimal dependancies on other workers completing theiur tasks. [It is really amazing to see]. There is no "specialized" technology involved.

Compare this to the average home which takes 2-6 months to build....

If someone was to approach a "conventional" builder, hand them a blank check, and tell them that they had 168 hours to COMPLETE (including furnishing and decorating) a project, 99.999% of them would be completely incapable of performing the task.

It goes directly back to what I said at the beginning of the post....their mindset and approach is simply not suited to the task at hand.

This is why I am disappointed (but not at all suprised that the MAJORITY of people who have responded to the poll answered "I dont know how". I really wish I could sample THAT category and ask if they had completed an education in programming at a school or other institution.....

[Joeman]

I think the os should handle all the splitting of processes and threads onto the appropriate cpu.

The os has way more leverage than the coders for processes do. It has many processes it can offload and not just little to big threads, so it should manage the cpu executions for all cpus.

if you have 60 active process, why shouldn't the os split up the processes and redirect execution of these processes on seperate cpus while trying not to maximize one cpu out while another one doesn't get any use. I find it hard to believe that every coder out there will have to manage this issue. the amount of cpus on a single cpu isn't going to stay the same. The only good ideal is to let something have full control over all processes and let it deal with what gets to what cpu.

[ahoodin]

Like, when I create new thread, does OS pick a core on which it is run?
Hyperthreading is a big help here.

In the past people had to use an API to start a thread on a processor other than 0.

With Hyperthreading, threads are started on the core/processor deemed by the system.

What I am saying is if you app is currently multi-threading and Hyperthreading is turned on in your bios, you are taking advantage of multiple processers/cores.

[Joeman]

So I vote not to worry about threading unless you do lots of loops that can be broken out of sync. I will wait until c++0x threading library comes out to consider it further :D

[TheCPUWizard]

So I vote not to worry about threading unless you do lots of loops that can be broken out of sync. I will wait until c++0x threading library comes out to consider it further :D

That (along with Ahoodin's post) is there part that does get me "upset" (when seen in professional developers).

First, there is no reason to wait for the ISO standard implementation of tools that help (but do not magically accomplish) parallalization; there have been very good libraries available for YEARS (consider Intel's offerings as an example).

Second, this does NOT address the fact that nearly every "real-world" application can significantly beneift from taking advantage of all of the processors (hyper-threads, cores, cpus) that are available.

Third, the entire philosophy of arctitecture and implementation is radically different. Unless this is taken into account now (or ideally 3-5 years ago), it will basically mean an entire scrap and re-write of the application (which most companies will not be willing to do, thus perpetuating the under utilization of the hardware).

As a trivial example, consider the following sequence:

1) Perform I/O to aquire data
2) Do calculations that must be sequential on the data in the order it is received
3) Perform calculations on the results that "could" be done in parallel.

First, lest assume that #2 is "fast" compared to #1... this leads to the first improvement (which is already carefully common

1) Initiate I/O as asynchronous blocks
2) As blocks become available perform sequential calculations
3) Wait for I/O (and calculations to complete
4) Perform calculations on the results that "could" be done in parallel.

Simple "overlapped I/O" (although it may be implemented using other techniques).
Now lets work on the last step...The way most people approach it (and which every library and language implementation (present and near future) I know of) will take this is:
code]
1) Initiate I/O as asynchronous blocks
2) As blocks become available perform sequential calculations
3) Wait for I/O (and calculations to complete
4) Spin up "processing paths" (threads, or fibers, or other constructs) to handle distributed work.
5) Perform independant calculations on the individual paths
6) Wait for all paths to complete.
[/code]
Better (probably) but still not taking into account the "philosophy".

a) During steps 1-3 above our premise (which would of course be supported by measurements for any actual development) is that the CPU(s) will NOT be at 100% capacity, I/O will still be the controlling factor.

b) Any method of spinning up "processing paths" will have overhead (e.g. creation of threads, initialization of state).

So the application would be better designed as...

code]
1) Spin up "processing paths" (threads, or fibers, or other constructs) to handle distributed work at a lower priority, having them block when fully initialized.
2) Initiate I/O as asynchronous blocks
3) As blocks become available perform sequential calculations
4) Wait for I/O (and calculations to complete
5) Wait (if necessary) for all "processing paths" to become ready
6) Perform independant calculations on the individual paths
7) Wait for all paths to complete.
[/code]
This type of design requires a much more "global" view of the applicaiton as a whole, and it is very unlikly that we will see programming environments which can automatically adopt a "holistic" approach.


If there is this much of a difference in structuring a simple path, imagine scenatios such as...

a) A data intensive application where the user will typically perform some operations and then generate a report...if processing power is available, why not generate the report dynamically in the background so that from a users perception, the process becomes "instant"?

b) A word process application that does spell checking on an existing document....Why not split the document up and do the checking on multiple threads...sure it will involve duplicate checks and more "total" processing, but if tuned to use available resources it will result in sghorter times

c) An "Explorer" (not IE, but desktop) that pre-fetches the adjectent directories (ie those most likely to be navigated to) and puts a "watcher" on them?

Compared to the list of applications that could not benefit from multiple execution paths, this list is virtual infinite.

[laserlight]

This is why I am disappointed (but not at all suprised that the MAJORITY of people who have responded to the poll answered "I dont know how". I really wish I could sample THAT category and ask if they had completed an education in programming at a school or other institution.....
Not yet, and I have not taken any modules with a focus on parallel processing, though I certainly have taken several that touch on the topic, but not enough for me to dare claim that I know how to design for it (and I probably would not confidently make such a claim even after I graduate).

I would have covered more of this as a hobbyist, but the current lack of support for parallel processing in standard C++ is something of a hindrance, though I am partly to blame for my reluctance to embrace more of Boost and POCO.

[ahoodin]

That (along with Ahoodin's post) is there part that does get me "upset" (when seen in professional developers).

Alright what did I do?

Sorry to be OT, but I thought I would respond to a question raised in the thread.

[ahoodin]

So I vote not to worry about threading

I vote that people think for themselves.:cool:


unless you do lots of loops that can be broken out of sync.

Huh? This part of your sentence is not so hot and could use a better explanation.

I will wait until c++0x threading library comes out to consider it further :D
Why wait when you can do it with a worker thread, using proper synchronization.

[Joeman]

Since I use manly c++, I worry about the compiler optimizing things to which will produce race conditions. Sure you can use violate, but what if you forget it? You will be so confused and it will take you hours to debug. Also I know how to write for different threads and I don't like it. Perhaps intel provides a cleaner way, but I am sure intel won't provide it for free. I like to keep my money if I don't need to spend it.

Multi-Threading your app can be very dangerous and time consuming with little payoff, if you app really isn't for critical processing time. Let's not mention on how sloppy your code can get when trying to pass things off onto others cpus.



unless you do lots of loops that can be broken out of sync.

Huh? This part of your sentence is not so hot and could use a better explanation.


I am saying keeping your variables in sync is a hard task due to race conditions, so it is best if you don't have to worry about such a mess... like not trying to sync your data with mutexes because even mutexes can become unsync due to unpredicted conditions. Once you prefect your sync, sure you can access your variables okay...


Why wait when you can do it with a worker thread, using proper synchronization.


Because the compiler can do things you can't do... and will never ever beable to do. I am waiting for the ATOMIC template...... wonder why?

[TheCPUWizard]

Alright what did I do?

Sorry to be OT, but I thought I would respond to a subtopic of the thread.


1) Hyperthreading as a specific technology has Nothing to do with it. In fact hyperthreading provides much less performance improvement than additional cores.

2) Using some actual application benchmarks
Single Core no-HT @3.2GHZ = 3.8 seconds
Q6600 Quad @2.4 GHZ = 1.8 seconds (theoretical best = 1.27 seconds)
i7 920 - Quad with HT @3.0GHZ = 0.85 seconds (theoretical best 0.51 seconds)
[/code]
This is on code which has not had structural changes in over 5 years.

This clearly shows that a core is better than a hyper thread..but more importantly.

As a consumer who just spent approx $1000 on a system upgrade, would you rather see a 6% performance decrease (because the developer didnt make best possible use of available processors) or a 4.4x (440%) increase

[I]What will YOUR customers experience when they upgrade their hardware to a Kiefer [32 cores!!!] and are running the CURRENT version of your software?????

[ahoodin]

1) Hyperthreading as a specific technology has Nothing to do with it. In fact hyperthreading provides much less performance improvement than additional cores.


Ok point taken here. I am seeing it now.

[ahoodin]

Since I use manly c++, I worry about the compiler optimizing things to which will produce race conditions. Sure you can use violate, but what if you forget it?

A. You can turn compiler optimizations off.
B. You can use volatile to make sure it doesn't optimize parts out.

I am saying keeping your variables in sync is a hard task due to race conditions, so it is best if you don't have to worry about such a mess... like not trying to sync your data with mutexes because even mutexes can become unsync due to unpredicted conditions. Once you prefect your sync, sure you can access your variables okay...

I am sorry but mutex, and semaphore are used successfully across platforms. This argument is just false.

[Joeman]

Sure mutexes work...... like i said... only if you perfect them.. i did mention about violate...

who wants to turn optimization off? omg... that is like really stupid. you missed the point. you want slower?

[laserlight]

i did mention about violate...
I think this is a miscommunication because you both mean volatile instead of "violate" or "voletile".

[ahoodin]

Sure mutexes work...... like i said... only if you perfect them.. i did mention about violate...

who wants to turn optimization off? omg... that is like really stupid. you missed the point. you want slower?

You missed my point. The point is that compiler optimizations can be dealt with properly. And I am sorry but there are alot of successful multi-threaded applications out there. Just because you didnt write one and don't know how still doesn't make this correct.

[Joeman]

I do know how they work and I do know when to use them, but 93% of code can't be offloaded... easily.

If you start naming the apps that use multi-threads, you will find out they need all the speed the can get... like video processing, physics and etc.

Most apps won't need to worry about that unless they bottleneck one processor for more than 4-15 secs.

[TheCPUWizard]

I think this is a miscommunication because you both mean volatile instead of "violate" or "voletile".

I like to use the "violate" keyword so that the compiler will randomly generate code that is non-compliant. :D :D :wave:


Sure mutexes work...... like i said... only if you perfect them..

There is no need to "perfect" or do anything to a mutex unless there is a bug in the compiler/library.

If by chance you meant "you have to use them correctly", then that applies to EVERY aspect of programming regardless of language, platform, threading, or anything...

Fundamently it is no different than sawing that aaddition and assignment only work if you properly hold the shift key (or not) while typing them in... :eek::eek:
(and yes, I have actually had that condition slip into code that made it to the QA department :blush::blush:)

[Joeman]

Yeah I meant volatile :D

Yeah I meant by the programmer not messing up mutexes since they can get difficult to use properly. Every newb will screw it up many times before they get the ideal where the fault is coming from. That is what I meant by perfect. I also would recommend using CreateMutex and not trying to create your own :).. and alone forget volatile :(

[Arjay]

Honestly, I don't see the big deal in programming for multithreading/parallel processing.

Sure if you explicitly manage the locking and unlocking of the synchronization objects (like cs or mutex), it can be a bit tedious, but if you create some simple helper classes and use RAII, it can be very straightforward.

[TheCPUWizard]

Honestly, I don't see the big deal in programming for multithreading/parallel processing.

Sure if you explicitly manage the locking and unlocking of the synchronization objects (like cs or mutex), it can be a bit tedious, but if you create some simple helper classes and use RAII, it can be very straightforward.

At a strategic level (large work items, a number of worker threads) it is not a "big deal".

At a more granular level, it becomes much more of a thought process.

Consider a simple application: There is a List structure with 8,000 lines of text. You need to determine which lines contain combinations of 8 key phrases. And produce the list of all combinations of string according to which of the 256 possible sets. You are on a machine with 8 "real" cores [2 CPU x 4/per]

Would you split the list into 8 chunks of of 1,000 strings?

Would you process the same string on 8 different threads (one per phrase)?

Would you take some other approach?

[Arjay]

At a strategic level (large work items, a number of worker threads) it is not a "big deal".Agreed, but for some reason folks still tend to have alot of problems doing it.

I hear quite a bit about people who find mt coding too hard, or don't see the benefits over st programming.

[TheCPUWizard]

Agreed, but for some reason folks still tend to have alot of problems doing it.

I hear quite a bit about people who find mt coding too hard, or don't see the benefits over st programming.

I see you "ducked" answering the questions about a specific scenario in my previous post... :rolleyes: ;)

[Arjay]

I see you "ducked" answering the questions about a specific scenario in my previous post... :rolleyes: ;)No interest in answering.

[GremlinSA]

Well I've Voted Yes (past 6 months).. My current project is using mutiple cores, however only small sections of the application does so..

Most of the application is linear and user speed dependent. but there are pieces that are CPU slaves .. IE ..

Once a transaction has been completed, the following steps need to be completed

1) Store Info Via Serial on a token.
2) Save Transaction Info in Sql DB..
3) Print Reciept.
4) Adjust sales stats (sql)
5) reset for new transactiion

Well 1 - 4 can all be done together, #5 can be done when all are complete (or more specifically when #1 is done)

My development system has a duo core, and starting a few worker threads at this point does show a marked increase in 2nd core usage..

At the moment this is the only muticore code that i'm using, Implimented 2 months ago..

Gremmy..

[dglienna]

Each can be done at the same time, if you send each task to a different message pump, running on multiple processors as available. The print receipt queue would only process when something was ready in the queue.

[TheCPUWizard]

Well I've Voted Yes (past 6 months).. My current project is using mutiple cores, however only small sections of the application does so..

That is at least a start, but what I have been referring to is MUCH finer grained. Lets look at the sample steps you posted and see what potentials there may be.

Important disclaimer: Since I am using Gremmy's sample some of these are "contrived" and most likely would not make sense from a practical standpoint using current common hardware (but may make sense within a few years). I am going to pretend that this work is being done on the server side and that small improvements in time would have an impact that would not exist for a client application where a human was waiting...
Lets look at the "Store Transaction into DB" step. This involves (typically)
[indent]
1) Create a Command Instance
2) Create Parameter Instances
3) Prepare the Command (often implicitly)
4) Create a connection (from a pool)
5) Execute the command
[/quote]
What would happen if there were alreat a number of (blocked) threads which had a command object already created and waiting for parameters. Then when it comes time to execute, the thread was awakened, and in parallel established the parameters whil the connection was being "refreshed" then executed the command, returned the result,and at a lower priority "freshened" itself before blocking.

Or the printing the reciept, where multiple threads were queued and ready to do formatting of the appropriate sections/fields...

---
On a 4,8,16 core machine (all currently available) these would (potentially) provide a benefit. But more importantly by controlling the number of threads processing specific types of work, the potential for blocking on a resource can be greatly reduced.

This approach is very much in keeping with Scott Meyers comment from the early 1990's of "Program in the Future Tense". The current approach to parallel computing seems much more to be "I dont need it now, so I will not worry about it till I do". IMPO, this will result in many software application being delivered TODAY that will not effectively make use TOMORROW's hardware without a complete system re-write.

[GremlinSA]

CpuWiz..

I get exactly what your saying here..

After reading this thread i was looking at the project and found at least 7 other places where i posibly speed things up a little, by using multiple cores..

IE... Checking contents of previous mentioned Token ...

1) Open View Form.
2) Reset Treeview (used to show the 7 different groups of info stored on Token.)
3) Read info from Token
4) Parse and Load in to Treeview.

This could also use a pre-prep'd (blocked)thread to proccess #3, while #1 & 2 proccess in the main thread .. at which time in the main thread we wait for the token read to complete, and then do #4....

The Pre-Prep'd thread used in #3 would acctualy get alot of work, as 90% of the project relys on reading and writing these tokens..

Comming from a VB background, I never really worked with threading, and now with .NET a lot of doors have opened..

Unfortunatly with the very tight schedule that i'm sitting on, i dont have enough time to play with an idea, and see it all the way through to the end...

However i do plan to take the time when the project is done, to further study multi-coring my applications..

[dglienna]

Did a project last week using Powershell 2.0 CTP. :thumb:

I had to rename a file if it existed, then copy over a 100MB+ sized file.

Every time I ran the script, I kept getting an error on the copy line, after the rename line.

I tried setting flags to hold it back, but it kept crashing, saying PROCESS IN US.

I woke up the next day, after thinking about it. PS must be using threads.

I searched the docs online, and found a -force parameter on the rename line.

It created the lock that I was trying to code the night before, and everything worked well.

[JohnW@Wessex]

My current project would be an ideal candidate for parallel processing across multiple cores. The project looks for similar objects in four areas of the image. The same code base is used for each area and therefore it should be possible to farm each one off to a core. Our spec is highly time constrained and this could be useful in extracting more processing time for each area from the system.

Unfortunately this is not going to happen until I get the chance remove all the 'static' variables the previous coder used in the analysis code. He had the habit of never to "Program in the Future Tense". :mad:

On the theme of synchronisation, we don't use any mutexes/critical sections in our code as this can lead to priority inversions which muck up the carefully crafted timing and thread priorities. We generally rely on a combination message passing, state machines and positional tracking (on the conveyor) to ensure non-concurrent read/writes to data.