Building Modular Applications with MEF

Pop quiz time. How many of you employ methods such as IoC in your projects, or for the non-devs, how many of you use an application that has some kind of 'Plug-ins' that it can use? I'm willing to bet that the positive result of that question is quite high, and as for the plug-in aspect, you quite likely don't even realise that you have an application that uses plug-ins.

Take a classic example, Adobe Photoshop. When you go to load a file, you get a tremendous list of file types the application can load. It may surprise you to learn, however, that natively Photoshop is only capable of loading its own PSD format files. If you dig around in the Photoshop folder, eventually you'll find a folder called 'Plugins' and in that folder, a further folder called 'File Loaders' (or something similar), so how is this magic achieved?

Each Plug-in is implemented as a DLL, and each DLL has the same calling structure. By calling structure, we mean that each DLL has exactly the same set of API/Method calls exposed, and that it presents any static information it holds in exactly the same way.

But .NET Does This Automatically with Reflection, Right?

Kind of. Reflection is a by-product of the way assemblies work in .NET. It's used by the run time to know what a file holds and what 'Stuff' it makes available. That 'Stuff' doesn't always have to be runnable code. It could be image resources, XAML libraries, or even custom binary data.

Because the whole concept is that library modules describe themselves to the runtime, this concept is central to the way .NET works. The difference here is that .NET looks for this information ahead of time. Plug-in operation is a slightly different ball game. With Plug-ins, the concept is to load when needed and not before.

For example, if you link a library for parsing CSV files to your project, you might need your code to be aware of that library at compile time, so that the compiler knows best how to pass the CSV data back to your application for a single use; that would be reflection that would provide that info. On the other side of the equation, you may give a user a choice of importing CSV, TSV, XML, or JSON, but you won't know until the application is up and running which of those schemes the user will choose. In this case, it's your Plug-in approach that then becomes responsible for obtaining that library based on the users choices.

In Windows API terminology (not .NET), the difference is known as 'Early Binding' versus 'Late Binding', and it's a very useful thing to have.

ModApps
Figure 1: The differences between Early Binding and Late Binding

First off, it means that you don't have to exhaust your memory by loading everything in one go, something which can be very useful in today's mobile world where devices may have considerably less memory available than many desktop or server environments. It also means that if you have an application that needs to remain running, you often can just change a simple DLL and have the app re-load it, a task which, due to the nature of early binding, often cannot be achieved without halting the application while the module is replaced.

What Part Does MEF Play in All This?

MEF is the .NET answer to making this late binding scenario work. It's a .NET API that allows you to look for external modules in various different ways, then include them within your application.

In some ways, yes, it is kind of similar to an IoC container, but there are some differences. In an IoC container, the module is generally found and loaded by the runtime code, in a similar scenario to Early Binding. Usually, too (at least in all the IoC containers I've used), it's a one off load; that is, you can load only ONE instance of a 'Foo Object' at once, whereas with MEF you can load several. Lastly, with IoC the loading is usually based on injected variable declarations that are automatically searched and loaded from a list built by some kind of bootstrap code, whereas MEF typically loads its modules based on being asked to look for anything that matches a given interface at a given time in the application's life cycle.

A Good, Practical Example

For the rebuild of the Lidnug.org website, we needed a data mining service that would be potentially be infinitely expandable. The purpose of this service would be to follow a given schedule, that would go off into the internet, obtain some data pertaining to Lidnug's community activities, and then make that data available to the Lidnug website.

Because most of our data comes from various social networks, we decided to build a core Win32 service that would periodically look in a defined folder in its installed location and load in any DLLs that it found in this folder, that followed a given Interface. Each DLL would be responsible for obtaining its own data, processing it, and saving it in our database. The DLL also would make available information on its task, author, and be responsible for tracking how often it runs.

Because social networks change all the time, we wanted to be able to just create a DLL, drop it in the folder, and wait a few seconds for the service to pick it up and start using it. MEF turned out to be the perfect choice for us to do this. Sure, we could have used standard file IO, and some reflection calls of our own to do this, but the code is long, messy, and very prone to errors. MEF handles all the dirty work for us, and makes things as easy as 123.

Like an IoC-based concept, we start with an Interface-based contract that defines what our Plug-in modules will do, something like the following:

namespace ModuleContracts
{
   public interface IPluginModule
   {
      string ModuleName { get; }
      string ModuleDescription { get; }
      string ModuleAuthor { get; }

      void ModuleStart();
      void ModuleStop();
      void ModuleTick();

   }
}

Nothing special here. We define three strings that will hold the modules information, and we stub out three methods that will form the body of the Plug-in, in this case a 'Start', 'Stop', and 'Tick'. The Start method will be called when the module is initialized, the stop when it's shut down, and tick will be called by the parent service on a regular time period defined in the service.

The tick routine will decide, based on the module's current state, if it needs to actually do any work or not returning. If not so, the next module can be processed.

To define a module that would A) Match that type and B) make itself available for MEF to find and use, we simply do the following:

using System.ComponentModel.Composition;
using ModuleContracts;

namespace TwitterModule
{
   [Export(typeof(IPluginModule))]
   public class TwitterMiner : IPluginModule
   {
      public string ModuleName{get { return
         "Twitter Data Miner"; }}
      public string ModuleDescription{get { return
         "Lidnug Data Mining plug-in module."; }}
      public string ModuleAuthor{get { return
         "Peter 'Shawty' Shaw"; }}

      public void ModuleStart()
      {
         // Implementation goes here
      }

      public void ModuleStop()
      {
         // Implementation goes here
      }

      public void ModuleTick()
      {
         // Implementation goes here
      }
   }
}

Looking at the code, you'll see there's nothing especially specific about its structure. It's just a regular class library (built as a standalone DLL) that implements the 'IPluginModule' interface. Anyone who's spent any time at all in .NET will immediately recognize this approach. The magic here, as far as MEF is concerned, is the Export Attribute applied to the class:

[Export(typeof(IPluginModule))]

You and the fact that we make use of the 'System.ComponentModel.Composition' library, which is the .NET namespace where MEF lives. Just the addition of these two elements makes your class ready to be consumed by a MEF consumer that might want to use your library.

In the case of the Lidnug project, all we simply wanted to do was find every matching DLL type, load them all into one big collection, and then just repeatedly call Startup, Tick, and finally Stop on each one in turn. This was easily achieved by first adding the following declaration to the consuming code:

[ImportMany(typeof(IPluginModule))]
private IEnumerable<IPluginModule>
   _availableModules;

Again, nothing special or new here. It's simply an IEnumerable collection using our interface contract as the base type, and an 'ImportMany' MEF attribute so that when we activate the MEF system, it knows what to match to what and where. The next part of the puzzle is the MEF activation code itself; this was achieved with the following method:

private void InitialiseHost()
{
   var catalog = new AggregateCatalog();
   string myPath = Path.GetDirectoryName
      (Assembly.GetExecutingAssembly().Location);
   myPath += "\\Modules";
   catalog.Catalogs.Add(new DirectoryCatalog(myPath));
   CompositionContainer container =
      new CompositionContainer(catalog);
   container.ComposeParts(this);
}

We get a new Aggregate parts catalog. We then create a path to a folder called 'Modules' under the location the service is running from. We then ask MEF to use a Directory Searcher and to build a list of all exports that match imports, where the exports are the DLLs in the specified folder, and the imports are specified inside the currently executing assembly or 'this', as you can see in the compose parts line.

Once this method completes, the _availableModules collection will then have a list of any matching 'IPluginModule' types found. From this, you then can do things such as the following:

public void StartHost()
{
   foreach (var module in _availableModules)
   {
      module.ModuleStart();
   }
}

Because it's an IEnumerable collection, you can do all sorts of Linq queries on it; for example, to only find a specific module, and call its functions. Unloading the collection is easy. Simply dispose of it, call your MEF initialisation function again, and the list will be re-built anew with the current set of DLLs found. Put this on a timer, and your code will refresh itself according to that schedule,allowing your app to monitor for and respond to DLL changes.

If you'd like me to cover a subject on .NET that you want to know more about, or just want to pick apart something I've already written, feel free to leave a comment below or come hunt me down on the interwebs and call me to task. I can generally be found hovering around in Twitter as @shawty_ds.



Related Articles

Comments

  • acronym

    Posted by dwight on 01/12/2015 06:01am

    MEF is an acronym. what does it mean.

    • RE: acronym

      Posted by Peter Shaw on 02/19/2015 02:09pm

      MEF stands for "Managed Extensability Framework", there's also "MAF" which is a similar technology, much harder to use which stands for "Managed Add-in framework" and is used more for app-domains as opposed to regular application scenarios.

      • MEF = Extensibility, MAF = Security/Isolation

        Posted by Ben Stabile on 10/24/2015 11:36am

        Although MEF can be used for lightweight (naïve) add-in functionality, it lacks some important capabilities that MAF provides out-of-the-box: isolation, security, and versioning. Sure, an application can include custom code that handles these additional capabilities. But most implementations that I've seen lack the coherent pipeline architecture that allows add-ins to interact and communicate easily. In fact, they usually resort to web service style interaction (e.g. WCF IPC). That is NOT, in my opinion, an optimal solution. What we really expect from add-ins is "invocation" against interfaces, not "messaging" across the wire. Supporting a robust and flexible add-in architecture, one that can safely accommodate third-party components, is not easy. But most of the work of setting up a MAF solution is incurred up front. After the initial pipeline has been hammered out, the addition of new add-ins is fairly painless. BTW: MEF, MAF, Unity DI, and whatnot, can all play together just fine. One just has to keep the separation of concerns in mind. Even Visual Studio 2015 still uses MAF for the Workflow Designer (although all new integration should be accomplished through VSIX packages).

        Reply
      Reply
    Reply
Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

Most Popular Programming Stories

More for Developers

RSS Feeds

Thanks for your registration, follow us on our social networks to keep up-to-date