Memory Size

[lt_w]

Hi,

I am porting a UNIX C mathematical system to Visual C++ for a research at school. Sometimes the system needs to locate large memory space by "malloc()" functions. My machine has 128M memory. For some reason, the memory (fault) swap is kicked in after 25M of the memory is allocated by the system. I add another 128M RAM to the machine, and then it pushes the limit to 50M. But my project eventually will require 200M space (even more).

Microsoft people told me to use function "setprocessworkingsetsize" to set the physical memory size. But I got error message code 120, which says that function is not supported in the system! I have installed the complete version of Visual C++ 6.0 professional edition on Window 98. They didn't answer why the function is not supported.

They refer me to Bugslayer Column on October 2000 MSDN Magazine about WST. Then I know the general purpose of WST. Be honest with you I have no idea about the content of the column, since I am new to Visual C++. I download and run the "smooth" program. And I get a link error of cannot open file "DBGHELP.lib". I guess I need to copy that from Window 2000 as mentioned on web. But I have a feeling that this is not a right prescription for my case.

I have more than enough main memory space. And what I really need is to set the physical memory size (or working set ?). Because it is really the data area triggers the memory swap. If I don't have a large working set, no matter how the functions are arranged in memory, the problem remains. Does my theory sound correct?

What should I do?


Li

[Vladdy]

Yes you're right. I have some programs that requires at least 25Mb, but my Win2000 says that my Virtual memory is low or refuses to load the program (but i have 64MB of physical memory). I guess the problem is memory management in Microsoft Operating Systems. I don't think you can do anything from your code. Maybe problem could be solved by doing smth in the Operating system.

Vladdy

[nikb]

Hello,

If you had taken the 5 minutes required to read the help file for the "SetProcessWorkingSize" function in MSDN, you would have noticed the following message on the Requirements section: Windows 95/98: Unsupported. The function is only available on NT 4.0 and Windows 2000 as of this time.

But onwards... I don't understand if the problem you're having is simply that data is being paged to disk (swapped) or that malloc() fails.

If malloc() fails, then perhaps your program does something which causes it to fail. Remember, that a lot of pieces of code that did not fail in UNIX can potentially fail on Win32.

If the problem you're having is that Windows swaps memory out, then you have little or no control over that. How Windows manages is memory space should not concern you. The data will be available to your application when it is requested. Remember, your application is not the only thing running on the machine, and Windows has to balance what you want, and what "MYCOOLAPP.EXE" wants...

-n

[lt_w]

Thanks for the information. I don't think the "malloc" failed, since the program is still running. But with disk swap, it is extremely slow. Any suggestion to confront that problem.

[nikb]

There is no way to control how windows swaps your process out.

I don't know how your application is supposed to work and what it is supposed to do, but a program that is supposed to grow to 200MB on a 128MB system will always swap. It will swap even if you have 256MB.... Windows won't let your process gobble up all the RAM.

I guess you could try doing the processing in chunks, to limit how much memory you use, and that is the best solution I can give you. There is little else you can do.

Sorry I could not be of more help,

-n

[Tom Lessing]

Windows NT and 95/98 treats memory a bit differentially. In general you don't care how windows allocates its memory it should be transparent to you. However there is a bit of difference in how the different versions handle this.

95/98 offers you very little control over the memory management. NT allows you to lock specific pages into memory using VirtualLock and a host of other thingies like SetProcessWorkingSize.

I suggest that you try the VirtualAlloc function. This offers you somewhat better performance on large chunks(bigger than 2 megs) (According to the Microsoft docs).

I recommend moving your project to NT.In my experience doing all kinds of stress tests with NT and in 95. NT comes out top. It just seems to tolerate stupid scenarios where you write a program to create 500 or a 1000 threads.

[nikb]

Hello,

First of all, I was under the impression that the "NT outperforms 9x under stress" was a well known fact, which is why I did not mention it. But it may be good to note it anyways.

Now, the problem this person is having, is Windows swapping parts of his process out, because he allocated large chunks of memory.

On 9x, there is absolutely nothing he can do to fix this problem. On NT, he could, theoretically, lock the pages in memory, however, I have found that the system tends to perform poorly when the memory managed is limited by locked pages.

One thing to point out, is that in the debug version of an application, the memory allocation code has extra overhead, to help developers track leaks. Moving to the release build memory allocator, he will probably see performance improve.

-n