Application Security Testing: An Integral Part of DevOps
- If you're experiencing memory problems (leaks/illegal memory access), mpatrol is for you (or your developers).
- There are other libraries out there, but they cost a fortune. Mpatrol provides an effective, no-cost, open-source alternative.
- Changes to your code base are minimal; for basic memory reports, no changes are required; a simple relink of your Visual Studio solution with the mpatrol library will suffice.
- Mpatrol supports various platforms. This article comments on Windows usage and provides binaries for Visual Studio to help you get started.
- See mpatrol's homepage for further information.
IntroductionWhat is dynamic memory allocation? Users of modern programming/scripting languages may have forgotten (or never learned) that, back in the old times, you had to keep track of all data structures you dynamically created and manually free them again when they are no longer used—or, you could forget about it (by evil intention or graceful ignorance), and your program would keep consuming memory until the operating system would refuse to cooperate (which could have been after a few minutes or several weeks, depending on your program's appetite).
The unfortunate among us who have to work with dinosaurs such as C/C++ (and are developing/maintaining non-trivial programs) need to make constant use of manual dynamic memory allocation/deallocation. In C, the *malloc()/free() family of functions will serve your allocation/deallocation needs.
In C++ you have the new/delete operator, making life a little easier because objects referenced from one object may be deleted automatically when said object is destroyed. I will not comment on this because there are enough excellent tutorials on the subject.
Now, where does mpatrol come in? Maybe you are an excellent hacker already, with strong analyzing skills and a near-perfect design. That will make your life a lot easier; nonetheless, even the wizzes (anyone know the plural of wiz?) who work on the Web browser Mozilla have their own set of tools to analyze their codebase and root out memory problems. So did Graeme Roy for his projects, and the result was mpatrol, an open source (LGPL-licensed) analysis tool for memory usage.
What does mpatrol do? Essentially, every memory (de-)allocation function (malloc(), calloc(), the new operator, and so forth) is provided by a runtime library. Mpatrol wraps these functions, so when you call them you will call mpatrol's customized versions. For that to work, you need to tell your compiler to use mpatrol functions, not the default ones. In most cases, relinking in a special way will already do the trick.
Will you need mpatrol only if you have a memory leak? Not necessarily. Mpatrol will tell you a lot about how your application is working with memory, thus serving as a profiling tool as well.
Preparing Your Visual Studio Project/Solution
Second, go to your project properties/configuration manager. Create a new configuration based on the one you want to work on. Give it a useful name; for example, if your configuration is called "MyProj Debug", call it "MyProj Debug - mpatrol".
Third, adapt your configuration as follows:
- Linker/General: Add the path where you put matrol.zip's contents.
- Linker/Input/Additional dependencies: Add "mpatrol.lib" as the first item in the list.
- Linker/command line/additional options: Add "/FORCE:MULTIPLE".
- Make the compiler produce as much symbolic information as you like.
- Optional last step: C/C++ / General/Additional include directories: Add the path where you put mpatrolSrc.zip's contents.
- Build your solution.
UsageThe above steps already provide you with an mpatrol-enabled executable. When run this executable, it will log the diagnostic output based on what is specified in mpatrol's environment variable.
Because it is a hassle to always set the environment variable the right way, there's another way as well: Run /path/to/mpatrol.exe <desired log output> <MyExecutable> in the directory where your executable is found.
What <desired log output> you specify depends on your needs. For leak detection, my command line looks like this:
C:\tools\mpatrol\mpatrol.exe -C --show-unfreed --leak-table MyApp.exe
What about the mpatrol.h file? Mpatrol can provide more details if you include "mpatrol.h" in your codebase, but that is entirely up to you. For a first analysis, those details may not be required. If you do include mpatrol.h, place the include at the very bottom of your includes.
Interpreting the Log FileThe "--leak-table" option will give you a table of functions that allocated memory and did not free it. This table is descendingly ordered by leaked memory, so the top entries are the most serious ones. If the top function is called from other functions in the table, it will only be reported once; you will see easily from the call stacks at the end of the log who the caller was.
A sample leak table looks like the following:
top 37 unfreed memory entries in leak table: bytes count location -------- ------ -------- 17469368 96 alloc2d 2676012 603 init_slicepar 794304 2758 create_new_widget 450252 14 FirstInitDataInterface 344000 1000 init_one_swip [...] [more entries here] [...] 22136522 5940 total
In most cases, the reported location in the code where the unfreed reference has been allocated is not the actual troublemaker; more often than not, some other code simply overwrote the reference, so it got "lost" and cannot be freed later.
Another issue is "Singleton" allocations: Many developers know that they need certain objects throughout the application's lifetime, so they do not bother to free them before exiting the application, knowing that the operating system will do that anyway. I only can heavily recommend to free those resources anyway: At some point, you cannot remember all those exceptions you made, and being accurate serves as another validation of your design—if you know exactly when to free each and every one of your objects!
Further ReadingSee mpatrol's home page for a detailed tutorial on usage and architecture of the library. The section on the Windows port may be especially interesting to you.
For C++ developers, I can recommend the boost library, providing numerous "intelligent" templates for memory management.