High Performance Solution Ini File Class with MMF

Environment: VC6, WinNT/2k/XP (Having been tested in English/Japanese Win2k/XP)

Special Note: This Unicode Version is full-fledged, while the ANSI version does not support the section header map

Key Technology Used: Memory Map File (MMF), Win32 Memory Management API, Unicode, Data Structure

Applicable Article Category in CodeGuru: Miscellaneous Data Technique

Summary

All is from my previous project, which saves the data of the coordinate, text contents, color, font name, and size information of nearly 30000 dialog controls (listbox, static, edit, and so on) distributed on more than 100 windows. So, the total size of these controls will be at least 10*n Mega bytes and it's not suitable to use the Windows Registry. You see, using a native Ini file Win32 API is terribly impractical because the performance is too low. So, I designed and implemented the following Ini class using MMF; with the pre-allocated buffer and section header map technology, it should be the ultimate solution to the 5-50 Mega byte level Ini File reading and writing situations.

Implementation Description

Pre-allocated buffer

Please refer the following figure:

Click here for a larger image.

When I load the Ini File from the disk to the MMF, depending on the case, I allocate a bigger MMF than the original disk file. When the user just reads the Ini MMF, the Pre-Buffer (see the red bar part) will not be touched. Say, if the user writes more data into the Ini MMF, I can expand the Ini MMF into the Pre-Buffer part. Only when the Pre-Buffer is nearly used up (I call it a Critical Point, see the slim green line), I unload the whole MMF to flush the disk, and load it again with a larger MMF. When measuring carefully in your case, you can get a fairly good performance upgrade with this extra buffer.

Section head position map

When loading an Ini File from the disk file, I make a map of Section Head Pointer and the Section Name. This will take time, but later, when the user searches a section in the end part of the Ini MMF file, it will save considerable time. In my Ini File implementation, I used the CMap MFC class, so this class needs MFC library support. Because my previos project is an MFC-based application, it is reasonable, but to be more universal, I plan to implement it again later with STL.

Special Note with Unicode Ini File:

It is very important to know that Unicode text files have two formats in the Windows OS. And what makes things worse is that Microsoft changed its behavior when it issued Win2000 SP1. In the original Windows 2000, if you open an ANSI text file with Notepad and save it as Unicode text, it just calls the MultiByteToWideChar WinAPI to make changes and save the text; that's it. After you install Win2k SP1 and do the same thing, this time, Notepad adds heading 0xFF FE to the Unicode text. This lack of congruity belabored me a whole day because some member's machine in my team did not upgrade with Win2k SP. (BTW, some specification on the Internet says that in Mac machines the heading will be 0xFE FF.) My Ini File class will check the heading and cope with it.

Usage:

I just want to kiss it (keep it simple, stupid). Unlike some previous Ini File classes on this site, I use just two functions that have the same signature of the Win32 API—GetPrivateProfileString and WritePrivateProfileString to wrap my Ini File class. For example, instead of adding a RemoveSection function, you pass NULL to lpAppName in JXWritePrivateProfileString and you get it. When you want to get all key names in a given section, pass NULL to lpKeyName and get a double NULL ending buffer in lpReturnString. (I include a full-fledged demo project to show this.)

DWORD JXGetPrivateProfileString(
  LPCTSTR lpAppName,          // section name
  LPCTSTR lpKeyName,          // key name
  LPCTSTR lpDefault,          // default string
  LPTSTR lpReturnedString,    // destination buffer
  DWORD nSize,                // size of destination buffer
  LPCTSTR lpFileName          // initialization file name
) ; 

BOOL JXWritePrivateProfileString(
LPCTSTR lpAppName,            // section name
LPCTSTR lpKeyName,            // key name
LPCTSTR lpString,             // string to add
LPCTSTR lpFileName            // initialization file
);

Both support ANSI and Unicode versions. Just include JXIniFile.h in your file and add both the JXIniFile.h and .cpp to your project. Check MSDN to get the detailed information because they are the same signature as the Win32 API.

There's just one thing I decided to be different from the Win32 API. When you want to get all the section or key names, and you pass a buffer to JXGetPrivateProfileString, a return value of 0 means your buffer is too small, while the Win32 API will return the buffer size needed. So, when you get the return of 0, just reallocate your buffer and try it until you pass. I did this because the program usually knows the rough size it takes to get the list of section/key names, and a mid-way return 0 in my Ini File class saves time.

Another thing is that I permit the user to unload the Ini MMF at any time by calling

BOOL JXUnloadPrivateProfile();

One possible use of it is that when you debug the program, you can unload the MMF, use a text editor to change some value, and ask your program read the Ini disk file again to see the effect.

Demo

I provide a simple, dialog-based program to show the usage of these two functions. The code to enumerate sections and keys is used to read a Ini file; usually, the program has knowledge of the section name and key name. I enumerate all the section and key names before fetching the value, just to give a worst-case example.

You press the buttons from left to right, generating 100 (by default) sections, each including 100 keys, the value is randomly generated; then, writing the data to disk, reading it back to the right side tree to confirm. The popped-up time message box tells you the total time to read the Ini File and fill the tree. (You can comment the tree-filling code and see the time of reading exclusively.)

Are You Ready To Use It?

1. If your application does not use MFC, you have to link with the MFC library because I use CMap to implement the section head map. I am considering implementing it again with STL to make it more universal.
2. If your application is not Unicode based, by default you will use its ANSI version. In that case, no section header map support is available. (Just because I have no time and need to use it.) It means the performance is not as good as its Unicode counterpart, but still better than the Win32 API (at least >> 10 times faster). Besides, you can still explicitly call JXGetPrivateProfileStringW and JXWritePrivateProfileStringW.
3. Consider your data structure. Instead of using 10 sections each including 1000 values, 100 sections with 100 values will be better.
4. Decide your pre-buffer size. I use 1M by default; maybe it is overkill in your case. At least it should keep MMF from reloading in a few seconds in most cases. Actually, if the program just reads the Ini data, you can use 4K (1 memory page).

Acknowledgements

I would like to express my thanks to the following persons and institute: Prof. Moriya Shinji, Human Interaction Lab of Tokyo Denki University, Media Center of Tokyo Denki University, for their academic direction, warm-hearted help, and continuous encouragements to my career life.

Also thanks go to FujiSoftware ABC, Ltd., for the developer position offered, plus their wage.

Downloads

Version History