Class CMaleLogger - Logging System

Environment: VC6 SP4, NT4 SP3, (Win32), Win95, Win98, Win2k, WinXP.

Introduction

Sorry for my bad english in advance. It is not my native language.

The CMaleLogger class is the child of the big project called 'Radionizer' that brings a complete automation to radio stations and it consists of several programs. The name of the main program in this project is 'JingleMale'. So, the class name has derived its 'Male' prefix from the program's postfix. The main reason for the birth of this class was I couldn't find any suitable logging system that's fast(!) enough, safe, stable, thread safe(!), easy-to-use(!) and self-sufficient(!). The one of the unique property of this class is that you may have only one instance of it to perform all tasks you really need. Generally, the only preparation to start using it is to declare its instance. That's all.

How Does It Work?

It is suprising, here is the only header file to bring powerful logging system to your application. And it needs to be just included in the files where your declaration or an external declaration is needed.

Why only the header file? This way all member functions are declared as inline versions. So, it's the chance for you that compiler will satisfy this request and the member functions will work as fast as possible. Actually, I don't like too many needless files scattered around! As I said, you have to declare the global and/or the only instance of the CMaleLogger class. There are many ways to declare and to use this class. The way you use it is up to you and your imagination.

The class is asynchronous. It essentially supports multidirectional log storage system (a single instance provides logging to different files independently).

The main logic of the class is to add log messages as fast as possible to the internal message queue and then periodically to save messages to proper files.

The CMaleLogger class provides easy-to-use, powerful, extra low CPU consuming and thread-safe logging system to multithreaded and single-threaded programs. It creates global-named mutex object to synchronize different threads and even different applications(!) that use CMaleLogger class. It is safe! When the class cannot store some message in some file (i.e. not now accessible to write) it doesn't stuck the rest of the queue for other files and postpones the store process to the next time for this inaccessible file. Even if the file will be inaccessible for a long time the m_OverflowLimiter will prevent queue from unlimited growth. It tries to keep the newest messages alive by removing the oldest ones within m_OverflowLimiter range. But if you provide your own overflow callback function then it is your responsibility in taking decision what to do. The class is flexible. Really, the only necessary action to start work is to declare the instance of the class properly. That's all.

General usage:
Include 'MaleLogger.h' header file where it is needed.
I. Common simple usage of thread-depended class instance:
Define file prefix like g_FilePrefix="C:\MyApp_" to get files look like "C:\MyApp_[2002-03-07].log"

  1. Global definition: CMaleLogger g_Logger;
  2. Call g_Logger.AddLog(g_FilePrefix, "Some log message...") at your code where logging is needed.
  3. Call g_Logger.Process() class member function periodically inside your own thread.

II. Common simple usage of stand-alone (self-sufficient) class instance:
Define file prefix like g_FilePrefix="C:\MyApp_" to get files look like "C:\MyApp_[2002-03-07].log"
  1. Global definition: CMaleLogger g_Logger(100, "log", NULL, NULL, true);
  2. Call g_Logger.AddLog(g_FilePrefix, "Some log message...") at your code where logging is needed.

There are special overloaded versions of AddLog and AddTimeStampedLog member functions which let you work with strings stored in resources.

If you intend to use the only one file to log in or just to type less symbols in your code then you may define macro commands to facilitate programming, like:


#define LOG_1(Message) g_Logger.AddMessage(g_FilePrefix,
                                           Message)
#define LOG_2(Message) g_Logger.AddMessage(IDS_RESOURCE_STRING,
                                           Message);

and just use it in your code like LOG_1("Some Message..."); LOG_2("Some Other Message...");

The constructor has the following form:

CMaleLogger(int iOverflowLimiter=0,
    CString iFileExt="log",
    bool (*ifpHeaderCallback)(CString &iLogHeader,
      const CString &iLogPath,
      const CTime &iHeaderTime)=NULL,
    bool (*ifpOverflowCallback)(const CFileException *fe,
      const HANDLE ihProcessThread,
      int &iOverflowLimiter,
      int &iUpdateFrequency,
      CString &iLogPath,
      const CStringList *iQueuedMessages,
      const CString &iFilePathTimeStampFormat,
      const CString &iMessageTimeStampFormat)=NULL,
    bool iUseInternalThread=false,
    int iUpdateFrequency=3000,
    CString iFilePathTimeStampFormat="[%Y-%m-%d]",
    CString iMessageTimeStampFormat="[%d-%m-%Y (%H:%M:%S)] ")

int iOverflowLimiter=0 It keeps the newest messages alive by removing the oldest ones within its range when overflow occurs. This situation could be caused by a file inaccessibility to write for a long time or by a too low update period of time for calls of Process() member function. This parameter prevents the internal queue from unlimited growth. It is some kind of memory care.
There are no limitations by default.
CString iFileExt="log" It gives to log file the extension (empty string gives no extension)
"log" is the default file extension by default.
bool (*HeaderCallback)
(CString &iLogHeader,
const CString &iLogPath,
const CTime &iHeaderTime)=NULL
The pointer to a user defined callback function that is called by the class when a creation of a new file occurs. The main purpose of this callback function is just to provide the header string to write to the file pointed in the iLogPath reference. You may analyze this argument to make a decision what kind of header you intend to provide. Your header must be placed to the iLogHeader variable. iHeaderTime indicates the time when file is created. You may use it for time-stamps. If the function returns false the header won't be added. Never call any class member function inside this callback routine because it'll hang (named mutex already locked).
There is no user callback function by default.

bool (*ifpOverflowCallback)
(const CFileException *fe,
const HANDLE ihProcessThread,
int &iOverflowLimiter,
int &iUpdateFrequency,
CString &iLogPath,
const CStringList *iQueuedMessages,
const CString &iFilePathTimeStampFormat,
const CString &iMessageTimeStampFormat)=NULL

The pointer to a user defined callback function that is called by the class when the overflow of the internal queue occurs. All non-const argument references can be analyzed and changed to desired values. All const arguments are given for analysis. fe is given to determine why the file cannot be accessible (do not delete it). ihProcessThread handle could not be used to kill the internal process just to determine if internal thread exists. If you kill the process, you will not be able to assign ihProcessThread the NULL value and the class will misfunction. You may change the iLogPath value and return true to change the inaccessible file path to another one otherwise you must return false. If callback function returns false the class do nothing. It means, if you define a user callback function you take all responsibility for controlling overflow situations. By default (if no user callback function defined), the class removes a header item from the QueuedMessages list. It is your responsibility to call iQueuedMessages->RemoveHead() if it is the necessary action. Never call any class member function inside this callback routine because it'll hang (named mutex already locked).
There is no user callback function by default.
bool iUseInternalThread=false Tells the class to run its own internal thread for calling Process() member function periodically.
It doesn't run the internal thread by default. You may run it manually later by calling StartInternalThread() member function.
int iUpdateFrequency=3000 Tells the thread the frequency for calling the Process() member function in milliseconds.
The update frequency is 3 seconds by default.
CString iFilePathTimeStampFormat="[%Y-%m-%d]" This is the time-stamped postfix which is appended to a file name each time it is created. For format specification read help file on CTime::Format(...) MFC class member function.
CString iMessageTimeStampFormat="[%d-%m-%Y (%H:%M:%S)] " This is the time-stamped prefix which is inserted before each message. For format specification read help file on CTime::Format(...) MFC class member function.

Here is the live example of the header callback function from my project:

bool HeaderCB(CString &Header, const CString &Path, const CTime &Time)
{
  if(Path.Find(g_GS.m_ProgramLogPath)!=-1)
  {
     Header=Time.Format("JingleMale program log file.\n\\
			logging started at %d/%m/%Y (%H:%M:%S)\n\n");
     return true;
  }

  if(Path.Find(g_GS.m_StaticLogPath)!=-1)
  {
     Header=Time.Format("JingleMale static blocks log file.\n\\
				logging started at %d/%m/%Y (%H:%M:%S)\n\n");
     return true;
  }
  
  if(Path.Find(g_GS.m_ErrorLogPath)!=-1)
  {
     Header=Time.Format("JingleMale error log file.\n\\
			logging started at %d/%m/%Y (%H:%M:%S)\n\n");
     return true;
  }

  return false;
}

Here are some possible ways to declare instance:

  1. CMaleLogger g_Logger; //simple declaration

  2. CMaleLogger g_Logger( 100,
                     "txt",
                     HeaderCB,
                     NULL,
                     true,
                     1000,
                     "(%m-%d-%Y)_err",
                     "{%H:%M:%S} "); //full custom declaration
    
    (notice the space character at the end of message time-stamp format. It is for a space between a time-stamp and a message)

  3. class CJingleMaleDlg : public CDialog, public CMaleLogger
    this way CMaleLogger class data members and functions become members of your CDialog derived class. But before using it you need setup it properly like:
    CJingleMaleDlg::CJingleMaleDlg(CWnd* pParent /*=NULL*/)
         :CDialog(CJingleMaleDlg::IDD, pParent) , 
         CMaleLogger(100,"txt",HeaderCB,NULL,true)
    {
       ...
    }
    

    or

    CJingleMaleDlg::CJingleMaleDlg(CWnd* pParent /*=NULL*/)
         :CDialog(CJingleMaleDlg::IDD, pParent)
    {
       SetOverflowLimiter(100);
       SetFileExt("txt");
       SetHeaderCallback(HeaderCB);
       StartInternalThread();
    }

    the same way you can merge it to your CWinApp derived class. Then you may use ((CMyApp*)AfxGetApp())->AddLog(...) function to call
    its member functions from any place of your application. Also you can define any universal macro
    like #define MY_LOG(Message) ((CMyApp*)AfxGetApp())->AddLog(g_FilePrefix,Message) ... or some way else.


  4. You may declare instance as a member of your class (like CMaleLogger m_MyLogger). But in the constructor of your class you must/may setup the instance as in CJingleMaleDlg class demonstrated.

  5. ... your imagination!

You see, you can use CMaleLogger object the way you need and the way you want.

Some words about file postfixes and time-stamp prefixes. AddLog and AddTimeStampedLog member functions uses file postfixes instead of real file paths. The real file path is generated from the file postfix with m_FilePathTimeStampFormat and m_FileExt consiquently appended. This way CMaleLogger creates new files on each other day or month or hour or minute and so on depending on your time-stamp format. I can suggest one trick to use file postfix to simulate file name prefix. You can define CString g_FilePrefix="C:\MyLogFolder\" and redefine internal m_FilePathTimeStampFormat like "[%Y-%m-%d]_MyLogFile". Thus the full path will be generated in the following way: "C:\MyLogFolder\"+"[%Y-%m-%d]_MyLogFile"+m_FileExt. The result may looks like "C:\MyLogFolder\[2002-03-07]_MyLogFile.log".

void AddLog(CString iLogPath, CString iLogMessage, bool iUseTimeStampForFilePath=true, bool iUseTimeStampForMessage=true) used to add a new message in the queue. iLogPath is a file prefix. iUseTimeStampForFilePath used to indicate whether to append file time-stamp to file name or not. iUseTimeStampForMessage used to indicate whether to insert message time-stamp before the message or not. Also, you may use overloaded AddLog functions to work with string resources.

You may use void AddTimeStampedLog(CString iLogPath, CString iLogMessage) member function to add new messages that already contain time-stamp format. If iLogPath or iLogMessage don't contain time-stamp symbols they stay intact. Also, you may use overloaded AddTimeStampedLog functions to work with string resources. I can suggest one trick to use the message prefix to simulate the message postfix. Just call AddTimeStampedLog(g_FilePrefix+g_Logger.GetFileNameTimeStampFormat(), "Simulation :-) "+g_Logger.GetMessageTimeStampFormat());

Anyway, you must call CMaleLogger::Process() member function periodically to store messages to files. If you don't want to run its internal thread for calling CMaleLogger::Process() you have to call it inside your own thread (for example, if your application has enough threads you may relay calling CMaleLogger::Process() upon one of them).

CMaleLogger::Flash(bool IgnoreAccess=true) is used to flash all messages in the queue to files. There is no need to call it before exit from your application because Flash is called in its destructor. IgnoreAccess with value of true is used to clear the queue forcely despite a file accessibility, but false don't touch the messages for inaccessible files.

Almost all member functions has the default values. It is useful when you want to reset some parameters to their default ones (i.e. calling SetFilePathTimeStampFormat() with no parameters will reset m_FilePathTimeStampFormat to its default value of "[%Y-%m-%d]").

The meaning of the rest member functions is easy to understand if you look inside the 'MaleLogger.h' file.

You may use this code freely without any restrictions except you MAY NOT change the name of the class and the names of its member functions, variables and the name of the mutex object.

Feel free to e-mail me any comments, suggestions and bug reports.

----------------UPDATED----------------

Beside some corrections, I included the new function static BOOL MakeSurePathExists(CString iPath, bool iFilenameincluded=true, bool iIgnoreResult=false)
The job of this function is similar to API BOOL MakeSureDirectoryPathExists(...), but a little bit accurate.
iFilenameIncluded used to point that the iPath contains filename but not only path.
iIgnoreResult used to point that you don't care the returning result (it's used to fasten function execution a little bit if you know that the path will be created with no errors as usually).
This function is static and you can use it as a stand-alone function without class declaration.
I use this function to create file path if it does not exist and there is not an overflow situation.

Moreover, at KillInternalThread function now I use API TerminateThread(...) function instead of CloseHandle(...). It is safe to use it because internal thread is intercepted in the state when no internal resources are used by the thread (mutex object locked in KillInternalThread)!

--------------------------------------------

Downloads

Download demo project - 49 Kb
Download source - 3 Kb


Comments

  • The new (NOT UPDATED) CMaleLogger class will be posted soon.

    Posted by Legacy on 03/23/2002 12:00am

    Originally posted by: Arlen Albert Keshabian

    The new UPDATED CMaleLogger class will be posted soon that considers your requests...


    Best regards,
    Arlen.

    Reply
  • might want to consider

    Posted by Legacy on 03/21/2002 12:00am

    Originally posted by: Michelle

    So when does "FemaleLogger" come out? ;)

    Reply
  • Callbacks vs. virtual function

    Posted by Legacy on 03/20/2002 12:00am

    Originally posted by: Paul McKenzie

    Overall, the concept of this class is good. However, here are a few observations:

    a) You use callback functions instead of providing virtual functions. I would have expected your logger class to be a base class, and the callbacks to be virtual functions that can be overridden by the user as he/she sees fit. The problem that I have with your design right now is that you made your class a "one and all" class, when it should be a base class. I noted that you responded to one message by saying that you can create a CMaleLoggerBin class. If you used the virtual interface, you wouldn't need to do this -- let the user decide by inheritance what they would like. Of course the logger would do the basic stuff, and delegate any user-defined tasks by calling virtual functions.

    b) If you are going to use callbacks, it would be easier if you "typedef"-ed the function pointers. For example:

    bool (*fpHeaderCallback)(CString &iLogHeader, const CString &iLogPath, const CTime &iHeaderTime);

    ------------------------------------------------------------
    Could be replaced by this:
    ------------------------------------------------------------

    class CMaleLogger;

    typedef bool (CMaleLogger::*HeaderCallbackProc)(CString &iLogHeader, const CString &iLogPath, const CTime &iHeaderTime);

    class CMaleLogger
    {
    HeaderCallbackProc fpHeaderCallbackProc;
    //...
    };


    Now all you need to say is HeaderCallbackProc instead of wearing out your keyboard in several places in your code.

    c) The class is MFC based. If I'm writing a Win32 API application or DLL and don't want to or can't use MFC, I'm out of luck. I started to write the equivalent calls to the Win32 API and to the STL classes instead of the MFC container classes.

    Overall, it is a good class -- it just needs some improvements that I hope you will utilize in your next version.

    Reply
  • My Changements

    Posted by Legacy on 03/15/2002 12:00am

    Originally posted by: Oguzhan ERKAN

    I call StartInternalThread from an Export function as you do. And there is no memory leak...
    
    void StartInternalThread()
    {
    CSingleLock lock(m_pMLMutex, TRUE);

    m_hThread = (HANDLE) _beginthreadex(NULL,0,ProcessThread,(void*)this,0,&m_threadid);
    if (m_hThread==(HANDLE)-1)
    {
    //Do something about error...
    };

    }

    void KillInternalThread()
    {
    CSingleLock lock(m_pMLMutex, TRUE);

    if (m_threadid<1)
    return;

    int a = 0;
    while ((!PostThreadMessage(m_threadid,WM_QUIT,0,0)) && (a <20))
    {
    a++;
    Sleep(10);
    };


    DWORD m_result = WaitForSingleObject(m_hThread,20000);
    m_threadid = 0;
    CloseHandle(m_hThread);
    }

    Reply
  • To Arlen...

    Posted by Legacy on 03/15/2002 12:00am

    Originally posted by: Amn

    Hello :)

    Which of the comments i added do you find appropriate, and which you think are not suitable on your opinion ? I am interested in improving my log system too, since i am starting a new project which needs even faster logging, and i am open to all suggestions :)

    The current implementation of my log traces messages of 10 words approximately 1000000 times/sec, but i think i can take it up 200%-500% from there.

    I have been working on graphic applications before DirectX and OpenGL were publicly available, and that is i so desperately pursue an always faster code ;)

    Regards,
    Amn.

    Reply
  • !!!!!!! ATTANSION !!!!!!! (I RESOLVED THE BOTH PROBLEMS)

    Posted by Legacy on 03/14/2002 12:00am

    Originally posted by: Arlen Albert Keshabian

    Hi everybody!

    Today evening I reviewed my code and correct some errors.
    First, I reject the AfxBeginThread function and start using API CreateThread function instead! This way I solved DLL problem!!!!!
    Second, this way I resolved memory leak on application exit!!!!

    I'll update my code at this article soon.
    Please, wait awhile.

    Best regards,
    Arlen.

    Reply
  • Cool logger but .....

    Posted by Legacy on 03/14/2002 12:00am

    Originally posted by: Nyrup

    Hi,
    I really like this class, it is very flexible and easy to use.
    Unfortunately I think there is a memory leak. If I compile your demo in a debug version and run it, my Developer Studie reports a 112 bytes memory leak on exit? I can't find the problem but I think it has something to do with the way the internal thread is terminated.....

    Any information is highly appreciated.

    Best regards
    Nyrup

    Reply
  • DLLs problem (maybe that's the solution)

    Posted by Legacy on 03/13/2002 12:00am

    Originally posted by: Arlen Albert Keshabian

    Yes. that's the Microsoft's MFC problem.
    ... Microsoft tells to export one initialization function within your DLL and then to call it inside your own application right after the AfxLoadLibrary("...") call to create the CMaleLogger's internal thread (by calling StartInternalThread() inside your export function). Maybe this is the solution. I've not tested it yet. If any of you have, please tell me the result.

    Best regards,
    Arlen.

    Reply
  • DLL problem

    Posted by Legacy on 03/13/2002 12:00am

    Originally posted by: Oguzhan ERKAN

    Hi,

    I'm trying your class in my project but I have some problems when I put it in a dll. In StartInternalThread
    function doesn't come back from AfxBeginThread... ProcessThread is valid... The program freezes then...
    Can you give any advice?

    Oguzhan...

    Reply
  • !!! Just For Fun And For...!!! :-)

    Posted by Legacy on 03/12/2002 12:00am

    Originally posted by: Arlen Albert Keshabian

    Hi everybody!
    Today I've tested my CMaleLogger in the strange way. I've tried to direct logging process to the floppy diskette and I've won!!! There is a huge HUGE difference between using a standard method to log and that of mine. It is great when you don't have to care much about the speed of storage media :)

    Moreover, I've tested the speed of AddLog function. So... it gives 33455 log messages per second on my computer.
    The test has looked like:

    for(int Idx=0; Idx<100000; Idx++)
    g_Logger.AddLog(g_SpeedLogFile,"Fast log...");

    then i've counted the number of lines writen within one second.

    Best regards,
    Arlen.

    Reply
  • Loading, Please Wait ...

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Hybrid cloud platforms need to think in terms of sweet spots when it comes to application platform interface (API) integration. Cloud Velocity has taken a unique approach to tight integration with the API sweet spot; enough to support the agility of physical and virtual apps, including multi-tier environments and databases, while reducing capital and operating costs. Read this case study to learn how a global-level Fortune 1000 company was able to deploy an entire 6+ TB Oracle eCommerce stack in Amazon Web …

  • Live Event Date: August 14, 2014 @ 2:00 p.m. ET / 11:00 a.m. PT Data protection has long been considered "overhead" by many organizations in the past, many chalking it up to an insurance policy or an extended warranty you may never use. The realities of today makes data protection a must-have, as we live in a data-driven society -- the digital assets we create, share, and collaborate with others on must be managed and protected for many purposes. Check out this upcoming eSeminar and join Seagate Cloud …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds