The 2 Gig file limit.

[GremlinSA]

following my article about the 2 Gig file limit (http://www.codeguru.com/vb/controls/vb_file/directory/article.php/c12917/) i've never realy had to suffer with this limit, until recently.........

The Wife has been sending Joke mails back and forth to several of her e-mail friends, with out ever clearing the Sent items folder...

The file hit the 2 gig limit yesturday, (un-be-known to me).. And of course while trying to send a few more mails, Outlook Express found it could no longer add sent mails to the SentItems.dbx file.. It finished sending with only a "warning", with out details and description... and left the mails in the Outbox folder..

After trying for an hour to send it through she eventualy left it ... only to be informed that she had sent the same 2 emails to one person 15 times each.. (everytime she tried to "Send/Receive"....)

Today i decided to have a look at it, and almost imidiatly noticed the problem... (2975 mails in the Sent Items folder) file size = 2,147,426,304
bytes...

Killing the file returned everything back to normal..

So something to note... the 2gig file limit applys to almost everything, and unles specially written to open files larger than 2gig (simular to the module in the article), keep the files limited to just under 2gig...

Gremmy....

[egawtry]

If you switch to a filesystem that supports files greater than 2GB, you should be OK in most properly written programs. FAT32 supports only 2GB file sizes. Switch to NTFS.

-Erik

[dglienna]

He's talking about Outlook Express. I know the same problam exists with pst files. You have to export them or create a new file.

[egawtry]

Funny. You would think that Outlook Express of all programs would be able to handle large files.

Do you know it this is fixed with "Windows Mail" under Vista?

[dglienna]

I don't know if FIXED is the word, but they have managed to break the strict limit, but there is still a slowdown.

http://channel9.msdn.com/showpost.aspx?postid=116711

[egawtry]

Well, at least they don't have than limit on the full Office Outlook. I have almost 10 years of saved mail stored without a hitch. No idea of the file size. I knew when I had XP, but all the data files were moved when I was forced to switch to Vista. I believe the last time I saw them they were at around 8.5GB.

-Erik

[GremlinSA]

If you switch to a filesystem that supports files greater than 2GB, you should be OK in most properly written programs. FAT32 supports only 2GB file sizes. Switch to NTFS.

-Erik
This has nothing to do with FAT or NTFS.. (BTW this is with Win2000 on a NTFS drive)

The problem is with the development platform used to write the application.. VB6, C compilers, and a few others use 32bit integers to store general standard values. Acctualy in some, 32bit integers is the largest numeric value the platform can hold, and some very fancy Text code is needed to handle anything larger..

With the highest bit been used to store the sign, you only get 31bits to store the value.. (2^31 = 2gig). This value is now used to find the location in the file to read/write.. (you can read and write past the 2 gig limit , however only with a single block)..

Back when most of these applications were written, email's were no more than 4K in size,giving you a limit of 524K items, More than any email addict could ever get through..

Now with video mails been the biggest hotest thing ever, some mails are as large as 4m , giving a item limit of only 524.

This is more a small oversite by early developers, much like '640K is enough memory, we will never need more' when the 8086/8088 was first built.

Average data file sizes have grown exponetially over the last few years, expecially when HDD's are getting bigger and cheeper all the time. This exponetial growth was not predicted 10 years ago.

On a side note I tested on FAT file system long before developing the module, it can hold files of any size. The Allocation tables allow for any size file, The directory entry table will only allow the size to show upto 4G (DWord or 32Bits), how ever you can write much more than this limit with the right methods. The only side effect is that the size is not reported correctly and any Disk checking tools will try to 'Fix' the invalid file size..

[TheCPUWizard]

This is more a small oversite by early developers, much like '640K is enough memory, we will never need more' when the 8086/8088 was first built.

Historical NitPick....

While that is an often repeated quote, it actually came well after the fact. The 8086/8088 was designed to handle 20 bits of address bus, 16 times what the previous 8080/8085/Z80 could handle.

In the days of CP/M, the top 8K (typically, sometimes it was more or less) was reserved for the BootRom, this left 56K for general use.

When the powers that be were writing the operating system, they decided purely for marketing purposes. to set the boundary between "normal" and "special" at the 640K boundary (of the total 1024K maximum).

This allowed them to marked "10x" as a magic mantra.

Of course once the hardware started getting built around this (with everything under the sun attempting to use the upper 384K) the boundardy becames relatively "Fixed". But even then, since chips are NOT made to decimal sizes, machines quickly had "real" memory sitting aboive the 640K (the first PC's were actually shipped with 128K and quickly raised to 256K), and access to the "UMB" became all the rage.....

Now give me back the days when you could run a 24,000 student school district (not for programming, for district admionistration, student records etc) all on a machine with 16K (12-bit) words of memory.....

[GremlinSA]

Now give me back the days when you could run a 24,000 student school district (not for programming, for district admionistration, student records etc) all on a machine with 16K (12-bit) words of memory.....Those were the day's..

[dglienna]

I posted about the old IBM. It ran a credit bureau at an upper class suburb.
I guess the guy bought the business and the machine. He quickly replaced it with a Honeywell, when he discovered that there was no way to calculate the payment amount! People mailed in their 'card' to be fed into the system, and it had the amount due. The problem was that 'someone' owed money, but no easy way to figure out who.

This was a friend of the family, and he didn't look all that well when I discovered this in the 7th grade, while visiting the office