Working with Memory Mapped Files in C#

C# Tutorials

Memory mapped files provide a way to work with data stored in memory as if it were a file. This can help when you are working with massive volumes of data, or when the data is being shared between multiple processes. In this C# programming tutorial, we will examine how to use memory mapped files in C#.

Memory mapped files are a mechanism that allows developers to access the contents of a file as if they were part of your program’s memory. The file itself is stored on disk, but reads and writes are performed by mapping the contents into your program’s address space.

The main reason for using memory mapped files is efficiency: since all memory operations are done in-memory, there is no need to copy data back and forth between Random Access Memory (RAM) and the hard drive every time you need to read or write something from/to disk.

Another benefit of using memory mapped files is that accessing them does not require loading an entire .NET assembly (as required with traditional file IO). This means it can be used in smaller projects where reducing dependencies on external libraries is important.

Looking to learn C# programming in a course? Check out our list of the Best Online C# Courses for Developers.

What is a Memory Mapped File?

A memory-mapped file is a kernel object that maps a file on your hard drive to a location in the main memory and hence provides direct access to the underlying data. This means that you can read and write the data directly to a file without having to go through any operating system or file system calls. Memory mapped files can be used in many different ways, but they are commonly used to share data between processes or to allow direct access to hardware devices.

Memory-mapped files may provide significant performance benefits over direct disk access when dealing with massive amounts of data or large files. The Win32API has always provided support for working with memory-mapped files, but until the latest releases of the .NET Framework, you were constrained to writing your code in C++ or PInvoke to leverage memory-mapped files. However, with the recent versions of the .NET Framework, i.e., .NET Framework 4 or later (as well as all versions of .NET Core) to be more precise, you now have built-in support for working with memory-mapped files.

Why Do We Need Memory Mapped Files?

The key benefit of using memory-mapped files is performance; they allow you to skip reading data from disk and then loading it into memory, which saves time because there are no additional steps involved with accessing this type of file compared to accessing one on disk. By using memory mapped files, you can avoid expensive disk operations and instead work directly with the data in memory.

Memory mapped files offer great performance because they allow data to be loaded directly into the buffer without first having to parse the file on disk. This allows files to be read faster than if they were being read off your hard drive by accessing them sequentially, which means less CPU usage and less memory overhead since you don’t need as much RAM in order for your application to run properly.

While there is a bit of a learning curve associated with working with memory mapped files, the performance benefits are definitely worth it.

Another benefit of using memory mapped files is that they are portable. They can be accessed from any environment and do not require additional libraries. This makes them simple to use and reusable across different types of applications, making them an ideal choice for various scenarios.

Another advantage of using them is that multiple processes can access the same memory mapped file without interfering with each other; since each process has its own virtual address space, only one process will have access to any given memory mapped file at any given time unless another process opens another copy of that file.

That said, one of the biggest drawbacks is that memory mapped files can be difficult to work with, and they often require a fair amount of knowledge about how the operating system and the file system work.

When working with memory mapped files you must be careful to keep race conditions at bay. Memory mapped files are not a good choice for applications of all types.

Read: C# Tools for Code Quality

Persistent memory-mapped files are linked to a source file on your system’s disk. When you work with memory-mapped files, the data is saved to the disk when the last process that worked on the file completes its activity.

Non-persistent memory-mapped files are yet another type of memory-mapped files with no storage associated with them to store data. Unlike a persistent memory-mapped file, data in a non-persistent memory-mapped file is never persisted. Hence, non-persistent memory-mapped files are ideal for sharing memory between processes.

How to Program Memory Mapped Files in C#

To work with memory mapped files in C#, developers will need to add a reference to the System.IO.MemoryMappedFiles namespace. We can do this using the using directive as shown in the C# code example below:

using System.IO.MemoryMappedFiles;

Once we have done that, we can create a persistent memory mapped file using the CreateFromFile() method:

MemoryMappedFile memoryMappedFile = MemoryMappedFile.CreateFromFile(@"D:\Demo.txt");

We can now create a view accessor for this file using the following code:

using (MemoryMappedViewAccessor memoryMappedAccessor = memoryMappedFile.CreateViewAccessor()) 
{ 
      // Write your code to read from or write to the file 
}

The view accessor gives us read/write access to the contents of the file. We can use the Read() and Write() methods to make changes to the view. For example, we could write an integer value 100 at the byte position 250 using the following code snippet:

memoryMappedViewAccessor.Write(250, 100);

Once we are done working with the memory mapped file, we should close the view accessor and dispose of the memory mapped file object. The following code listing illustrates how you can create a memory mapped file and write the character ‘A’ into it:

using System.IO.MemoryMappedFiles;
using (MemoryMappedFile memoryMappedFile = MemoryMappedFile.CreateFromFile(@"D:\Demo.txt", FileMode.Create, "Demo", 1000))
{
    using (MemoryMappedViewAccessor accessor = memoryMappedFile.CreateViewAccessor())
    {
        accessor.Write(250, 65);
        Console.WriteLine("A new memory-mapped file has been created...");
    }
}
Console.ReadLine();

What are the Use Cases of Memory Mapped Files in C#?

There are a few different scenarios where using a memory mapped file can be helpful. One common use case of memory mapped files is when you are working with huge files where the data should be accessed in a random manner.

Another scenario is when multiple processes need to share data. You can take advantage of memory mapped files to create a shared memory area which can be accessed by multiple processes.

Memory mapped files can also be used for inter-process communication (IPC). For example, one process could use a memory mapped file to write data that another process could then read.

Read: Implicit versus Explicit Type Conversions in C#

What are Some Alternatives to Memory Mapped Files in C#?

One alternative to using memory mapped files is a managed cache, such as the System.Runtime.Caching.MemoryCache class. This can provide similar performance benefits to memory mapped files, but with the added benefit of automatic eviction of unused items from the cache.

Another alternative is to use a streaming API, such as the System.IO.Stream class. This can be a good choice when working with data that is constantly changing, or when you need more fine-grained control over how the data is accessed.

A database is yet another alternative to using memory mapped files. You can take advantage of a database as an alternative to using memory mapped files when you would like to work with vast large amounts of data.

Final Thoughts on Memory Mapped Files in C#

You can take advantage of memory mapped files for direct access to file data without having to go through the file system as well as for sharing memory between processes. If you are working with large files or need to share data between multiple processes, memory mapped files may be worth considering.

Read more C# programming tutorials and software development guides.

Joydip Kanjilal
Joydip Kanjilal
A Microsoft Most Valuable Professional in ASP.NET, Speaker, and Author of several books and articles. More than 25 years of experience in IT with more than 18 years in Microsoft .NET and its related technologies. He was selected as a Community Credit Winner at http://www.community-credit.com several times. He has authored 8 books and more than 500 articles in some of the most reputed sites worldwide including MSDN, Info World, CodeMag, Tech Beacon, Tech Target, Developer, CodeGuru, and more.

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read