You've Got the BizTalk POP3 Adapter

Email has become a ubiquitous part of every company's information technology infrastructure. Now, just about every software application provides some form of email support. Email was one of the first "killer apps," rivaled only by the word processor in its contribution to productivity.

The increased amount of email, though, led to a new productivity bottleneck. In particular, how does one manage the bewildering number of messages appearing in an inbox? Software to manage an inbox has led to new commercial opportunities. The deluge of email has given rise to tools for organizing, prioritizing, and filtering email messages. Some software even scans email arriving in a generic inbox and routes email to an appropriate individual.

Recently, my company developed a BizTalk solution for scanning and routing emailed integration result logs from hosted applications. In this article, I'm going to show you how we utilized the BizTalk Post Office Protocol version 3 (POP3) Adapter to inspect and route the incoming email messages.

Managing an Inbox

Integration log results we get from our Internet hosted apps must be reviewed daily for errors.

Most of the time, an integration process with one of these hosted applications is successful. Humans generally disdain and are not good at repetitive tasks. Reviewing a daily email of mostly successful integration logs is a repetitive task.

Although making a cursory scan of a log is hardly time consuming, a log is easy to dismiss, especially when a process is usually successful. Also, people take vacations and change jobs. Reviewing a mostly successful process generally gets forgotten in times of transition.

We could have assigned the job to our systems group, but here again you have the human factor to contend with. In addition, a business user is often better at assessing the impact of a failure.

Somehow, we needed to automatically check for error messages and notify a business user as soon as an error was found. At the same time, we needed to provide our business users with some means to manage the process. As you might imagine, there were a number of ways to attack this problem.

The Platform of Choice—BizTalk

Our firm supports two email systems, IBM Lotus Notes and Microsoft Exchange. We thought of writing a custom application triggered whenever an email message was received. We decided on BizTalk mainly because we wanted to take advantage of BizTalk's SharePoint integration and system workflow capabilities.

Rather than building a new Adapter or an Orchestration, we decided that a Custom Pipeline Component would be our best choice. Our solution resembled the figure below.

To implement the solution, we needed to add some additional information to the message's context and search a message's Stream object for a particular pattern. Because all operations occur when a message is received, we worked with the POP3 receive Adapter.

Configuring the POP3 Adapter

A complete introduction to the POP3 Adapter is beyond the scope of this article, so I'll focus on how we configured the POP3 Adapter for our solution. See the sources at the end of this article for more POP3 Adapter details.

Like other adapters, the POP3 Adapter translates data from a native format, MIME in this case, to an XML format palatable for any BizTalk application. As you can see below, relative to other BizTalk adapters, the POP3 Adapter has a small number of settings.

We used "Basic" authentication and most of the default settings, so all we needed to enter was a user name, password, and mail server. I'll talk later about the Body Part Index setting.

The POP3 adapter deletes messages from the POP3 mailbox once the message is moved into BizTalk. The POP3 adapter does not use a transaction when it pulls the message into BizTalk. So, the potential exists to receive the same message twice. This is a POP3 limitation and not a BizTalk shortcoming.

Messages emitted from the POP3 adapter can be derived from one of the following parts of an email message:

  • The body of the message
  • A message attachment
  • The header information of a message, including the subject and list of recipients

The Body Part Index controls which message part is emitted. You can access all parts from an Orchestration or a Pipeline. As stated earlier, our solution used a Custom Pipeline Component.

Custom Pipeline Component

A Custom Pipeline Component was a better choice than an Orchestration for the following reasons:

  • The overhead and workflow capabilities of an Orchestration were not required.
  • A pipeline has fewer working parts and dependencies than an Orchestration, thus making deployment and configuration easier.
  • An existing application allowed us to easily customize a Pipeline, adding custom code by simply adding a new assembly.

Understanding that a POP3 BizTalk message is divided into multiple parts was the key to implementing our Pipeline solution. The following code illustrates how you can access each one of the POP3 message parts using functions on the IBaseMessage inteface.

for ( int n=0; n<_bizTalkContext.BizTalkMessage.PartCount; ++n )
{
   partName = "";
   _bizTalkContext.BizTalkMessage.GetPartByIndex(n, out partName);

   PipelineChannelTrace.WriteLine("Part index == "
      + n.ToString() + " part name == " + partName);

   part = _bizTalkContext.BizTalkMessage.GetPart(partName);

   part.GetSize(out partSize,out implementedValue);

   PipelineChannelTrace.WriteLine("Part information "
      + part.ContentType          + " IsMutable=="
      + part.IsMutable.ToString() + " size=="
      + partSize.ToString()       + " size implemented?.. "
      + implementedValue.ToString());

   stream = part.GetOriginalDataStream();

   WriteStreamToFile(stream);
}

The preceding code relies on the code from my "BizTalk Pipeline Dreams, Part I" article. Parts of a message include attachments as well as the message header and body information. As you can see in the code above, ContentType and Size properties allow you to handle binary attachments such as Word or Excel files.

In our solution, some of the information populated by the POP3 adapter was not configured to be a "promoted" field; examples are the "From" and "To" fields. We also needed fields indicating the result of the pattern search. So, in addition to looking for patterns in the message, our Pipeline promoted some of these fields so we could route messages using Filters on BizTalk Send Ports configured with the BizTalk SharePoint Adapter.

The final part of our solution required searching for keyword patterns in the BizTalk message Stream.

Scanning the Message for Patterns

The technique we used to scan the Stream for patterns will be the subject of a future article, so I'll just outline the techniques we used.

We combined RegEx, the Regular Expression class, and string manipulation .NET classes in our message Stream scanning solution. Fortunately, we were dealing with ASCII-based attachments and an ASCII message body. Also fortunate for us, the patterns we were searching were simple regular expressions, such as the word "error."

We recognized that attachments and message body can be large. So rather than copying the data into a string and scanning the string, we copied small parts of the Streamed message into a buffer and scanned the buffer.

Conclusion

Email has been one of the biggest contributors to information worker productivity. The productivity gains have come with a price, though; in particular, information workers increasingly need better tools and techniques for managing their inboxes. We recently built an application from BizTalk and the .NET framework to move integration process log result messages monitoring out of users' inboxes and into a centralized automated process.

Sources

About the Author

Jeffrey Juday is a software developer with Crowe Chizek in South Bend, Indiana. He has been developing software with Microsoft tools for more than 12 years in a variety of industries. Jeff currently builds solutions using BizTalk 2004, ASP.NET, SharePoint, and SQL Server 2000. You can reach Jeff at jjuday@crowechizek.com.



About the Author

Jeffrey Juday

Jeff is a software developer specializing in enterprise application integration solutions utilizing BizTalk, SharePoint, WCF, WF, and SQL Server. Jeff has been developing software with Microsoft tools for more than 15 years in a variety of industries including: military, manufacturing, financial services, management consulting, and computer security. Jeff is a Microsoft BizTalk MVP. Jeff spends his spare time with his wife Sherrill and daughter Alexandra.

Comments

  • BizTalk POP3 Adapter does not download the attachments from Different Emails

    Posted by Ranjan Kumar Behera on 06/18/2012 09:38am

    Hi Jeffrey, Thanks a lot for your article. I have the same requirement to download the attachments from email; which is come from different mail Ids. I have designed an Orchestration which one downloading the attachments from some of the email ids and some of the email id, it does not download. It's throws the different error. One of the Error is: There was an authentication failure. "Failed to decode the S/MIME message. The S/MIME message may not be valid.". After read your article; I thought of write the custom Pipeline component to download the email attachments. Could you please help me on writing the custom component, else if you have some sample; could you please share the code. Thanks in Advance. Thanks and Regards, Ranjan.

    Reply
Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • IBM Worklight is a mobile application development platform that lets you extend your business to mobile devices. It is designed to provide an open, comprehensive platform to build, run and manage HTML5, hybrid and native mobile apps.

  • Agile methodologies give development and test teams the ability to build software at a faster rate than ever before. Combining DevOps with hybrid cloud architectures give teams not just the principles, but also the technology necessary to achieve their goals. By combining hybrid cloud and DevOps: IT departments maintain control, visibility, and security Dev/test teams remain agile and collaborative Organizational barriers are broken down Innovation and automation can thrive Download this white paper to …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds