Threads - how much time do they really use???

[radboudp]

Hey everyone,

I have an interesting problem. My application consists of multiple (about 9 in all) threads, all doing their own little part of the work that needs to be done.

The bulk of the work is done by two threads: One with high priority and one with low prio. Basically data is received by a thread, queued for the high prio thread, which does some work on it and queues it for the low prio thread, who in turn, does some work on the received data. As long as it is nice and calm, the data streams right trough. Great! Of course, when it would get extremely busy, the data would still stream like water through the high prio thread, but would heap up in the low prio thread. This is still as it is intended.

Now, for some reason, while it is still relatively slow, the data still starts to heap up in the low prio queue. The explanation is simple enough: One of the other 7 (or so) threads with normal priority is requesting a lot of time from the CPU and the low prio thread is not given enough processing time.

What I want to find out is which thread is causing my problem. Does anyone know good methods or tools to determine which thread is using/being assigned how much processing time???

Hope someone can help!

Laters,
Radboud

[JMS]

If you have multiple normal priority threads running why does it shock you that the time slicer would starve your lower priority threads in order to feed your higher priority threads?

Sounds to me that your problem is the nut behind the keyboard... Seriously.. Of course your higher priority threads will starve lower priority threads...

You know the way the cpu meter works? Make a low priority thread and have it loop for some time integrel.. (n ms)... then inversly graph the index of the loop.. as the cpu gets busy the loop index n will become very small and the inverse bar graph spikes....

Solution's... if you must use the priority thread only make something normal priority that runs infrequently and sleeps alot.. Better yet make everything normal priority and then you won't have the OS stepping on your programms coat tails either when it decides to run some obscure service...

7 threads in a program reminds me of that Howard Hughes fill ****'s angels... Howard used 27 camera's to film the final sequence... Howard was a famous obsesive compulsive... Don't you think he could have used 2-3 camera's if he really thought about what he was doing?

[Andreas Masur]

[ Moved thread ]

[MikeAThon]

...
What I want to find out is which thread is causing my problem. Does anyone know good methods or tools to determine which thread is using/being assigned how much processing time???

I don't have a direct answer to your question (sorry) but it sounds like your low-priority thread needs a dynamic priority boost. Look at functions like SetThreadPriority(), which your low-priority thread can call based on the size of its processing backlog. Also read about scheduling of thread based on their priority at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/scheduling_priorities.asp

Mike

[Mathew Joy]

I feel there is no need to tinker with the thread priorities. In general, high priority threads should be short (in duration) and infrequent(in execution). Thread priorities should be changed only after having a good knowledge of the basic functioning of the scheduler as well as the functioning of your app: changing the priorities (both thread and class) affect the whole progam as well as other apps that run on your system.

If you really want to go on with the priority thing, I suggest you do some dynamic priority boost if you see a thread is starved. Of-course it increases the complexity of the program as well.

[Andreas Masur]

If you really want to go on with the priority thing, I suggest you do some dynamic priority boost if you see a thread is starved. Of-course it increases the complexity of the program as well.
Well...this is already taken care of by the Windows operating system... :cool:

[kirants]

Probably, you may want to to yield to other threads in your 7 dwarfs !! You could use Sleep(0)

But, coming to your question, it may turn out to be that it may not be the 7 dwarfs that is the problem, it could be any of the threads in the system..

Also, you may want to use performance counters to check whats going on.

[Mathew Joy]

Well...this is already taken care of by the Windows operating system...There is Priority boosts, done by the windows os, increasing the dynamic priority of a thread. But, does that mean that any starved thread will receive priority boosts? Accordingly the priority boosts are given...
When a process that uses NORMAL_PRIORITY_CLASS is brought to the foreground, the scheduler boosts the priority class of the process associated with the foreground window, so that it is greater than or equal to the priority class of any background processes. The priority class returns to its original setting when the process is no longer in the foreground.
Windows NT/2000: The user can control the boosting of processes that use NORMAL_PRIORITY_CLASS through the System control panel application.
When a window receives input, such as timer messages, mouse messages, or keyboard input, the scheduler boosts the priority of the thread that owns the window.
When the wait conditions for a blocked thread are satisfied, the scheduler boosts the priority of the thread. For example, when a wait operation associated with disk or keyboard I/O finishes, the thread receives a priority boost.

[Andreas Masur]

Dynamic boosts

Each thread has a dynamic priority as well. This is the priority the scheduler uses to determine which thread to execute. Initially, a thread's dynamic priority is the same as its base priority. The system can boost and lower the dynamic priority, to ensure that it is responsive and that no threads are starved for processor time. The system does not boost the priority of threads with a base priority level between 16 and 31. Only threads with a base priority between 0 and 15 receive dynamic priority boosts.

[darwen]

Can I ask a simple question here....

Why doesn't he just have a normal priority for all the threads and just let the OS take care of it ?

I've ran a system with over 100 threads on a W98 system and it behaved fine.

Darwen.

[radboudp]

Hey guys,

Thanks for all your replies. Just to be clear: I am a pretty seasoned programmer. I have full understanding of how the thread scheduling in the Windows OS works and I know my app inside out. This is also not just a run of the mill small app. It does some pretty hard work and complex work. Hence the multiple threads. Most of the '7 dwars' as you guys have started calling them, do IO stuff, while there are two threads that do the processing. Because the IO is on multiple network cards and/or other devices, they need to be in seperate threads. (It wouldn't be smart to get something, perform operations on it, and send it of again within the same thread because you will be waiting for all that IO to complete!!)

But, coming to your question, it may turn out to be that it may not be the 7 dwarfs that is the problem, it could be any of the threads in the system..

Also, you may want to use performance counters to check whats going on.

I have found a great way to monitor the thread activity in a program, although it requires some code... First, make sure you have a handle for each thread. Then create a new thread in which you periodically call GetThreadTimes for each handle. This function returns the time that the thread has been allotted. By determining the delta (with the last measurement), logging it to a file and then graphing the result (MS Excel), you can see pretty clearly when which threads have been active and what effect that had on the other theads. I already know which one it is. Now I have to find a way to fix the problem!

Blow you can see the code I am using. The handles to the threads are stored in the class pointer that I have provided as a paramater to the thread function. It uses some of my standard functions that I have in my projects, so it won't compile. But you will be able to get the general idea and use it for yourself if you want to...


//---------------------------------------------------------------------------
// Monitor thread

bool CServiceS1::StartMonitorThread()
{
bool bSuccess = true;

m_bExitMonitorThread = false;

if ( m_hMonitorThread != NULL )
::CloseHandle( m_hMonitorThread ), m_hMonitorThread = NULL;

if ( (m_hMonitorThread = ::CreateThread( NULL, 0, MonitorThread,
this, 0, &m_dwMonitorThreadID )) == NULL )
bSuccess = false;

::SetThreadPriority( m_hMonitorThread, THREAD_PRIORITY_HIGHEST );

return bSuccess;
}

void CServiceS1::EndMonitorThread()
{
m_bExitMonitorThread = true;
}

DWORD WINAPI CServiceS1::MonitorThread( void *lpParameter )
{
CServiceS1 *pThis = (CServiceS1 *) lpParameter;

CString csLogLines;
time_t tLastLog = time( NULL );

csLogLines = _T("Main,MainDelta,Switch,SwitchDelta,S0Connect,S0ConnectDelta,ProcessS0,ProcessS0Delta,"
"ProcessLow,ProcessLowDelta,WinPCap,WinPCapDelta,S2Queue,S2QueueDelta,Monitor,MonitorDelta\n");

unsigned __int64 i64LastExecTimeMain = 0;
unsigned __int64 i64LastExecTimeSwitch = 0;
unsigned __int64 i64LastExecTimeS0Connect = 0;
unsigned __int64 i64LastExecTimeProcessS0 = 0;
unsigned __int64 i64LastExecTimeProcessLow = 0;
unsigned __int64 i64LastExecTimeWinPCap = 0;
unsigned __int64 i64LastExecTimeS2Queue = 0;
unsigned __int64 i64LastExecTimeMonitor = 0;

while (( ! pThis->IsClosing() ) &&
( ! pThis->m_bExitMonitorThread ))
{
pThis->GetTimesForThread( &i64LastExecTimeMain, pThis->m_hMainThread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeSwitch, pThis->m_hSwitchThread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeS0Connect, pThis->m_hS0ConnectThread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeProcessS0, pThis->m_hProcessS0Thread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeProcessLow, pThis->m_hProcessLowThread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeWinPCap, pThis->m_hWinPCapThread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeS2Queue, pThis->m_hS2TlvQueueThread, csLogLines, false );
pThis->GetTimesForThread( &i64LastExecTimeMonitor, pThis->m_hMonitorThread, csLogLines, true );

if ( (time( NULL ) - tLastLog) > 5 )
{
pThis->m_logMonThreads.Dump( csLogLines );
tLastLog = time( NULL );
csLogLines.Empty();
}

Sleep( THREAD_MONITOR_INTERVAL );
}

if ( ! csLogLines.IsEmpty() )
{
pThis->m_logMonThreads.Dump( csLogLines );
csLogLines.Empty();
}

::ExitThread( 0 );
return 0;
}

void CServiceS1::GetTimesForThread( unsigned __int64 *pi64LastExecTime, HANDLE hThread, CString &csLogLines, bool bNewLine )
{
FILETIME ftCreationTime;
FILETIME ftExitTime;
FILETIME ftKernelTime;
FILETIME ftUserTime;

CString csThreadTimes;

if (( hThread != NULL ) &&
( ::GetThreadTimes( hThread, &ftCreationTime, &ftExitTime, &ftKernelTime, &ftUserTime ) ))
{
unsigned __int64 i64ExecTime = (*(unsigned __int64 *) &ftKernelTime) + (*(unsigned __int64 *) &ftUserTime );
unsigned __int64 i64ExecTimeDelta = i64ExecTime - *pi64LastExecTime;

*pi64LastExecTime = i64ExecTime;

time_t tTime = (time_t) (i64ExecTime / 10000000i64);
time_t tFracTime = (time_t) (i64ExecTime % 10000000i64);

time_t tTimeD = (time_t) (i64ExecTimeDelta / 10000000i64);
time_t tFracTimeD = (time_t) (i64ExecTimeDelta % 10000000i64);

csThreadTimes.Format( _T("%d.%07d,%d.%07d"), tTime, tFracTime, tTimeD, tFracTimeD );
}
else
{
csThreadTimes.Format( _T("-,-") );
}

csLogLines += csThreadTimes;

if ( bNewLine )
csLogLines += _T("\r\n");
else
csLogLines += _T(",");
}

[radboudp]

Hi Darwen,

Why doesn't he just have a normal priority for all the threads and just let the OS take care of it ?

The answer to your question is very simple: The functionality of my application (which is actually a service doing background work) requires that all data received first is analysed for a specific 'characteristic' that needs to be 'responded to' IMMEDIATELY. Secondly most data needs to be processed for other purposses. However, this second processing can wait. (As long as I don't run out of memory storing the data. ;)) Hence two threads, one with high priority, one with low priority.

Grtz,
Radboud

[Andreas Masur]

Hence two threads, one with high priority, one with low priority.
Well...and what happens if there is nothing to receive? Unless the thread is in some kind of blocking, non-consuming CPU time state it will use CPU time...to more or less do nothing. Depending on the actual flow of data, it might be better to simply change the priority of the corresponding thread only if there is something to receive (while receiving higher, otherweise normal)...

This will give you a better utilization of the available CPU time...

[radboudp]

Well...and what happens if there is nothing to receive?...

dwWaitStatus = ::WaitForMultipleObjects( nEvents, ahEvents, FALSE, 250 );

Two events: One to detect if the app is closing, one that is set when data is added to the queue of data that needs to be processed... On the timeout the function that checks and processes the data from the queue is executed as well, even though this is quite useless (but not harmfull either)...

Unless the thread is in some kind of blocking, non-consuming CPU time state it will use CPU time...to more or less do nothing. Depending on the actual flow of data, it might be better to simply change the priority of the corresponding thread only if there is something to receive (while receiving higher, otherweise normal)...

This will give you a better utilization of the available CPU time...

Something like this?

static void Class::Thread( void * lpParam)
{
Class *pThis = (Class *) lpParam;

while ( ! exit )
{
pThis->ThreadWork();
Sleep(10);
}
}

void Class::ThreadWork()
{
SetThreadPriority( ::GetCurrentThread(), PRIORITY_ABOVE_NOMAL );

// Do work...

SetThreadPriority( ::GetCurrentThread(), PRIORITY_NOMAL );
}

My thread actually is triggered by an event

[MikeAThon]

...I have found a great way to monitor the thread activity in a program, although it requires some code... First, make sure you have a handle for each thread. Then create a new thread in which you periodically call GetThreadTimes for each handle.
Thanks for identifying this function (i.e., GetThreadTimes). BTW, I see it is only supported in W2K and above; you wouldn't happen to know of an equivalent W98 function (I need to support some older OS's).

This function returns the time that the thread has been allotted. By determining the delta (with the last measurement), logging it to a file and then graphing the result (MS Excel), you can see pretty clearly when which threads have been active and what effect that had on the other theads....
Actually, the function gives the amount of time that the thread has executed in kernel mode and user mode, not the amount of time allotted. But that seems to be what you want anyway.

As you mentioned, you need the delta to see high processing intervals. Do you do this in Excel (I didn't see it in your code).

And out of curiousity, do you see large amounts of time spent in kernel mode, or is most of it in user mode?

Mike

PS: If possible, can you post a sample graph (like I said, it sounds interesting).

[Andreas Masur]

Thanks for identifying this function (i.e., GetThreadTimes). BTW, I see it is only supported in W2K and above; you wouldn't happen to know of an equivalent W98 function (I need to support some older OS's).
As far as I know (actually as far as my brain thinks to know), there is no direct API to use in systems prior to Windows NT. However, you might be able to do something similar using VxD...but this won't be that easy and thus might not be worthwile...

[Mathew Joy]

The system can boost and lower the dynamic priority, to ensure that it is responsive and that no threads are starved for processor time.Well, yes. This is said in the MSDN from where I quoted. But it doesn't say below, that the dynamic priority boosts are given if a thread is found to be starving.

[Mathew Joy]

The answer to your question is very simple: The functionality of my application (which is actually a service doing background work) requires that all data received first is analysed for a specific 'characteristic' that needs to be 'responded to' IMMEDIATELY. Secondly most data needs to be processed for other purposses. However, this second processing can wait. (As long as I don't run out of memory storing the data. ) Hence two threads, one with high priority, one with low priority.Well, what is the point if the high priority thread needs the service of low priority ones to finish the 'important' work? If your app gets busy your low prio thread will starve or not get enough cpu time. But as you said, the bulk of the work is being done by it. I feel you can do better by effective communication between and synchronizing the threads without changing the priorities. For instance, your low prio thread can become a normal prio one and can do the process or yeild depending on the 'busy'ness factor of other threads. Hence whether to do the process or yeild will depending on your decision (dending on other factors at that state of your app), rather than the OS. So you have more control. For instance you could increment a variable, in an atomic fashion(using Interlockedxxx apis), when one of the 7 threads are busy and decrement if not. Your other thread can look at the variable and make a decision.

/Just a few tips/hints on what I understand.

[Andreas Masur]

Well, yes. This is said in the MSDN from where I quoted. But it doesn't say below, that the dynamic priority boosts are given if a thread is found to be starving.
Priority boosts can happen in five cases:

On completion of I/O Operations
After waiting for kernel objects such as events or semaphores
After threads in the foreground process complete a wait operation
A GUI thread awakes due to windows activity
A thread ready to run hasn't been running for quite a time (CPU starvation)

Note though, that priority boosts will not be given to threads in the real-time range (16 through 31).

[Andreas Masur]

Something like this?
Yes...I was referring to such a behaviour...thus, the priority of the thread is only increased when it actually needs to do work. This all of course depends on the frequency of the data flow. I use this a lot while dealing with our complex business software whichg gets triggered over network (but not necessarely with a static frequency)...however, the trigger needs to get processed as fast as possible. Thus, the thread will get real-time priority for the time of processing (which is only trigerring some connected hardware).