Building the Right Environment to Support AI, Machine Learning and Deep Learning
If developers want to leverage muti-cores, without a doubt they'll need to tackle Threading and race conditions, typically referred to as Concurrency. Concurrency is complicated. So, Microsoft has been working to make Concurrency more accessible to a wider range of developers doing things like embedding Concurrency features in their languages and building APIs around Concurrency.
Concurrency and Coordination Runtime (CCR) is one of Microsoft's Concurrency APIs available via download to MSDN subscribers. CCR is part of the "CCR and DSS Toolkit 2008 R2 Standard Edition". Although CCR was originally constructed for Robotics solutions the patterns it employs can be applied to a broader set of domains. In fact, if you're considering or currently using the .NET framework ThreadPool or Threading, you might want to check out CCR. Before you do, some guidance may be helpful, so, I'm going to show you how CCR can fit into your next .NET Framework Concurrency-dependent application.
.NET Asynchronous Programming Model (APM)
- There is a
Begin____(BeginRead, BeginWrite, etc.) operation. The Begin operation accepts a delegate that receives a "state" parameter and a result parameter.
- There is also an
End___operation. End is invoked inside the delegate passed to the Begin operation.
- There are typically 2 distinct execution results, the operation succeeded or the operation failed.
There may be some wrinkle to the pattern above, but that's the general APM idea. APM is important if a developer is building a responsive application. Rather than blocking main thread execution for an operation to complete, APM continues a threads execution and wakes some other part of an application up when an operation completes.
APM is typically used in conjunction with the Threadpool. I like to think of the Threadpool as being composed of three things:
- A Queue you place your function into.
- A set of spinning threads awaiting work.
- An object that assigns a function from the queue to an available spinning thread.
In a running application, at any time, multiple threads may be attempting to change the same place in memory (shared state). Alongside APM and the Threadpool a developer must synchronize access to shared state. A developer typically employs various locks and mutexes to signal exclusive access to state.
Briefly summarizing, a typical APM solution contains the following components:
- One of the APM "callback-oriented" APIs
- Threading and some sort of queue-like data structure to control thread usage
- Synchronized access to state.
All of these aspects and more are elegantly captured in the CCR data structures.
A complete introduction to all aspects of the CCR is beyond the scope of this article. So, instead, I'm focusing on the core classes and classes more closely associated with the APM solution I outlined above.
Main CCR classes appear in the graphic below:
Click here for larger image
Figure 1: CCR Main classes
Dispatcher works a lot like the ThreadPool. Dispatcher maintains a group of threads. However, instead of pulling from a single Queue, a Dispatcher may pull from multiple Queues (Dispatcher Queues). Dispatchers operate on their Dispatcher Queues round-robin style. Typically a Dispatcher is configured to create a thread for each core on the hosting computer. I'll cover more about the Dispatcher later in the article.