Adventures with _chkstk

WEBINAR: On-demand webcast

How to Boost Database Development Productivity on Linux, Docker, and Kubernetes with Microsoft SQL Server 2017 REGISTER >

_chkstk Routine is a helper routine for the C compiler. For x86 compilers, _chkstk Routine is called when the local variables exceed 4K bytes; for x64 compilers it is 8K.
That's all that you get from _chkstk()'s MSDN web page. Nothing more...

Overview

A process starts with a fixed stack space. The top of a stack is pointed to by the Extended Stack Pointer (ESP) register and this is a decrementing pointer. Every function call results in a stack created for the function inside this Process Stack. Every thread function has its own stack. The stack is a downward growing array. The default stack reservation size is 1 MB and the heap's size theoretically increases to a limit of 4 GB on 32bits operation systems. See more information here.

Every thread under Windows has its own block of contiguous memory, and while function calls are made, the stack pointer is increasing and decreasing. In contrast, a different thread within the same process might get a different block of contiguous memory – its own stack. When a context switch occurs, the current thread’s ESP (along with the IP and other registers) are saved in the thread’s context structure, and restored when the thread is activated the next time.

In order to specify a different default stack reservation size for all threads and fibers, use the STACKSIZE statement in the module definition (.def) file. To change the initially committed stack space, use the dwStackSize parameter of the CreateThread, CreateRemoteThread, or CreateFiber function.
Most stack problems occur in overflows of existing stacks, as their sizes are fixed and they cannot be expanded.

_chkstk() increases the stack when needed by committing some of the pages previously reserved for the stack. If there is no more physical memory available for committed pages, _chkstk fails. When you enter a function (VC++ with the stack checking enabled), it will call the _chkstk located in CHKSTK.ASM. This function does a stack page probing and causes the necessary pages of memory to be allocated using the guard page scheme, if possible. In this function, it is stated that when it encounters _XCPT_GUARD_PAGE_VIOLATION, the OS will attempt to allocate another guarded page and if it encounters _XCPT_UNABLE_TO_GROW_STACK then it’s a stack overflow error. When _XCPT_UNABLE_TO_GROW_STACK is encountered, the stack is not yet set up properly; it will not call the catch because calling it will use invalid stack variables, which will again cause another exception.

Case – Too Many or Too Big Variables on Stack

As I said above, the function stack size is 1 MB. If you miss that and you're trying to define and use an array internally like this:

int arr[4000][200];

when you compile with VC++ compiler in debug mode, you will have a big surprise: the application crashes on _chkstk() at the moment the _chkstk() tries to create a new memory page on stack and fails.

Call Stack
Figure 1: Call Stack

The output window shows the next message:

 
First-chance exception at 0x004116e7 in testApp.exe: 0xC00000FD: Stack overflow.
Unhandled exception at 0x004116e7 in testApp.exe: 0xC00000FD: Stack overflow.

This happens because the 1MB limit is overloaded even on a win32 OS: 4000*200*4 = 3.2MB (approx.). It's the same story if you define many local variables and their stack usage overloads the 1MB limit. If you really need this big array then use heap to avoid this crash.

 
int **arr = new int*[4000];
for(int i=0; i<4000; ++i) {
	arr[i] = new int[200];
}
 
// and finaly don't forget to delete
for(int i=0;i<4000; ++i) {
	delete[] arr[i];
}
delete[] arr;

Case – Recursive Functions

If you have an infinite recursion then you will gate same stack overflow error and the application crashes in _chkstk.asm. Recursive function is outside the scope of this article. Here is a good example of what happens with recursive functions.
The solution is to avoid using recursive functions as much as possible and try to implement an iterative function.

Case – Stack Corruption

I started looking over _chkstk() implementation the moment I got a few bugs with crashes that had similar details. I had to analyze some .dump files and solve a few bugs that contained a call stack with _chkstk() on top. Most of the .dump files call stack contained a second similar thing: the call of a thread function (so called ThreadFoo()) that was running in a threads pool. I started to research why _chkstk() failed and my first track was to debug the stack overflows. I followed a MSDN debugging tutorial and unfortunately I didn't find anything strange. I checked to see if the local stack variables were filling the ThreadFoo() function’s stack and they did not.

A new study of ThreadFoo() function followed, in order to detect internal function calls that can fail in some circumstances. I checked some of the trace function calls in great detail. Those trace functions where defined in an external debug class and each time a new trace file was added it used an internal buffer (TCHAR szBuff[2048] = _T(“”);). The writing of this buffer was done using: swprintf(). As we know, this function is unsafe and is not recommended. As long as the content of these trace lines was dynamically built (in some cases those lines may contain dynamically built SQL queries that fail) then the length of these trace lines could be higher than 2048 bytes; then guess what, a stack corruption appears! UPS! The stack pointer will be corrupted (the classic stack overflow case).

So I have implemented and used the following macros:

#if defined(UNICODE) || defined(_UNICODE)  
#define usprintf(x, ...) \
   _snwprintf(x, _countof(x)-1, ##__VA_ARGS__); \
   x[_countof(x)-1] = 0
#else
#define usprintf(x, ...) \
   _snprintf(x, _countof(x)-1, ##__VA_ARGS__); \
   x[_countof(x)-1] = 0
#endif

Now, if we use the safe macro, we will have no issues.

 
int x = 23;
TCHAR szMsg[2048] = {0};
usprintf(szMsg, _T("Error: %d"), x);

Another safe way is to allocate the buffer in the heap. However, in some cases, stack arrays may be preferred for performance reasons. After that was fixed I met with no other stack corruptions in ThreadFoo() or other code areas. Even if the top of the call stack was _chkstk() this was not the function that failed. The error appeared because of that stack corruption and _chkstk() has just detected.

Conclusion

If your code produces a stack overflow, then you have to rethink your design in the right away:

  • If you see _chkstk() on the top of call stack, check if you have no stack corruptions - stack overflow
  • Don't try to manipulate the stack by yourself. The default 1MB stack size is basically enough
  • Allocate dynamically if you're using big arrays
  • If you have recursive functions producing a stack overflow, re-write them using loops (a tip: it is a proven fact that any recursive functions can be programmed non-recursive)

Resources



About the Author

Silviu-Marius Ardelean

Silviu Ardelean is a software developer working for an American-based company. His main focus is on building C++ applications for Windows and Unix/Linux operating systems. Personal web log: www.silviuardelean.ro.

Related Articles

Comments

  • ADMIN: Commenting

    Posted by Brad Jones on 11/01/2011 10:49am

    We do everything we can to avoid removing comments; however, comments that contain negative statements aimed at an individual will be removed becaue they violate this sites Acceptable Usage Policy. You can comment on the code and the article, but please keep comments towards people professional. Thanks..

    Reply
  • interesting template function

    Posted by Maximus_X on 10/19/2011 03:43am

    "Actually there are already such functions in Windows"
    Indeed, but when I came with the idea of using them, from different reasons my boss decided to avoid the introduction of these functions after the middle of the current version lifetime (in the last 2-3 project iterations).  Itbs obviousb&

    Reply
  • An alternative to macros

    Posted by Viorel on 10/19/2011 03:13am

    Maybe it has sense to consider a template function too instead of your 'usprintf' macro? This is an attempt for Unicode strings: template< size_t SIZE > bool usprintf(wchar_t (&buffer)[SIZE], const wchar_t * format, ...) { va_list argptr; va_start(argptr, format); bool success = _vsnwprintf(buffer, SIZE, format, argptr) >= 0; if( ! success) buffer[SIZE - 1] = 0; va_end(argptr); return success; } It works with buffers allocated on stack and calculates automatically the bufferbs size. Actually there are already such functions in Windows.

    Reply
  • ADMIN: Offtopic: Please watch your comments.

    Posted by Brad Jones on 10/18/2011 12:08pm

    Please watch your comments. You can freely debate whether code is right or wrong and whether the article presents information in a useful way or not. What you cannot due is post negative comments about others. There is no place for comments like "how dumb can you be," "Crawl back under your rock...," etc. Discuss the technology, not the technologist. Please do not respond to this off topic comment. If you have a further issue you want me to address, freel free to contact me via the forum or via webmaster@ this site's domain name. Thanks. Brad!

    Reply
  • conclusions

    Posted by Maximus_X on 10/18/2011 07:37am

    "Instead of simply fixing the size by using _countof() at the location of the original call"
    1st using _countof() with swprintf() is not proper (out of the case you want to add the count value in buffer). 
    On the other side it's hard to add _countof() in a huge solution for each _snwprintf() and that's why we used some macros.
    
    "you now have a macro everywhere which the maintenance programmer will have to look up"
    The cost of a new team member looking over a simple macro like this is insignificant comparing with the situations when your product is crashing to customers (just because in some situation the buffer limit(2048) was . :)
    As I said in article, I started from a concrete situation when I had to analyse different .dump files and those files were created by issues like this. 
    
    "Did I understand it right now, or am I still missing something?"
    It seems you have a too small overview and I don't try to convince you any more as long as you don't know me in person but you dare to qualify me in different ways. 
    Any other discussion is redundant and out of topic.
    Have a nice day!

    Reply
  • finally...

    Posted by Maximus_X on 10/18/2011 06:18am

    I'm happy you finally understood the idea.

    Reply
  • Hahaha #2

    Posted by mihnea on 10/18/2011 05:42am

    Oh wait, I see what you're trying to do. You're trying to say that you knew all along what the prototype of swprintf() was, but that you were actually trying to prevent people from lying about the size of the buffer. Awesome. Simply awesome.

    Reply
  • @Mihnea Balta

    Posted by Maximus_X on 10/17/2011 04:16pm

    The functions used in the macro (_snwprintf/_snprintf()) are not proper to replace functions like vswprintf() - this function has different count of mandatory parameters and uses a list of arguments parameter. Sorry, in article I mean swprintf() witch is unsafe. Thanks for observation.

    Reply
  • msdn

    Posted by Maximus_X on 10/17/2011 05:51am

    "Using vsprintf, here is no way to limit the number of characters written, which means that code using this function is susceptible to buffer overruns. Use _vsnprintf instead, or call _vscprintf to determine how large a buffer is needed. Also, ensure that format is not a user-defined string. For more information, see Avoiding Buffer Overruns." Same story with vswprintf(). 
    http://msdn.microsoft.com/en-us/library/28d5ce15(v=VS.80).aspx

    Reply
  • Learn the meaning of flame

    Posted by mihnea on 10/15/2011 04:44pm

    Your "article" states:
    
    "The writing of this buffer was done using: vswprintf(). As we know, this function is unsafe and is not recommended"
    
    How is it unsafe and how is using _snwprintf() any better?

    Reply
  • Loading, Please Wait ...

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • The software-defined data center (SDDC) and new trends in cloud and virtualization bring increased agility, automation, and intelligent services and management to all areas of the data center. Businesses can now more easily manage the entire lifecycle of their applications and services via the SDDC. This Aberdeen analyst report examines how a strong foundation in both the cloud and internal data centers is empowering organizations to fully leverage their IT infrastructure and is also preparing them to be able …

  • Whether a mandate to secure all web and mobile apps comes from a newly enlightened CIO or in response to a major security breach, beginning even a small application security program can be a daunting task. How will you know how many digital assets you have, let alone their risk profile? In this webinar we explore how, using a cloud solution like Fortify on Demand, even the largest organizations can begin to scan apps immediately and rapidly scale an application security program. Identify and risk rank assets, …

Most Popular Programming Stories

More for Developers

RSS Feeds

Thanks for your registration, follow us on our social networks to keep up-to-date