File Too Large Python

I am writing to a file using python. The script suddenly stops running and throws an 'IOError: [Errno 27] File too large' Is there a limit on the size of the file that you are allo

Solution 1:

This Python bug report indicates that the OS is the source of this error message, not Python.

Since you are writing to a FAT partition, and the maximum file size limit is 4GB for FAT 32 (LinuxFilesystemsExplained) this is most likely the cause of your problem. Running your program on a system or partition with a different file system would tell you for sure.

Solution 2:

I also got this error when I had too many files in one directory. I had 64435 files in a directory, each with 10 digits + '.json' in their names, and any subsequent attempts to write new files to the directory threw errors (e.g.) OSError: [Errno 27] File too large: 'ngrams/0/0/0/0000029503.json'

Solution 3:

When files get too large, addressing becomes an issue. Typically you get 32 bits which translates to a maximum size of about 4 gb.