I am building a large data dictionary from a set of text files. As I read in the lines and process them, I append(dataline) to a list.

At some point the append() generates a Memory Error exception. However, watching the program run in the Windows Task Manager, at the point of the crash I see 4.3 GB available and 1.1 GB free.

Thus, I do not understand the reason for the exception.

Python version is 2.6.6. I guess, the only reason is that it is not able to use more of the available RAM. If this is so, is it possible to increase the allocation?

解决方案

If you're using a 32-bit build of Python, you might want to try a 64-bit version.

It is possible for a process to address at most 4GB of RAM using 32-bit addresses, but typically (depending on the OS), one gets much less. It sounds like your Python process may be hitting this limit. 64-bit addressing removes this limitation.

edit Since you're asking about Windows, the following page is of relevance: Memory Limits for Windows Releases. As you can see, the limit per 32-bit process is 2, 3 or 4GB depending on the OS version and configuration.