Cannot write more than 204800 characters in a file?

Question

EDIT 2 : For those of you who'd like to get a closer look at the code, here it is : https://github.com/pikzen/ffbookmark/blob/python-rewrite/ffbookmark.py

I'm having a bit of a trouble here : python throws an IOError whenever I try to write more than 204800 characters into a file. I tried on another computer, and it crashed at 768k characters. Is it a python problem, is the OS limiting something ? Here's the code I'm using :

with open('out.json', 'w') as f:
    json.dump(items, f)

items is a simple dictionary. I'm building it from an HTML file containing about 800 elements. Each element is built like this :

bookmark = {}
            bookmark["title"] = link.contents[0]
            bookmark["id"] = id
            bookmark["parent"] = 5
            bookmark["dateAdded"] = 1
            bookmark["lastModified"] = 1
            bookmark["type"] = "text/x-moz-place"
            uri = link.get('href')
            # Shaarli's self links are totally messed up : ?xGRpkrp
            # But we can't simply oust links containing '?'s, because
            # php uses it, and pretty much everything does
            # however, if there's not dot, we can assume it's a 
            # Shaarli link.
            # If it's not, well too bad, false positive.
            if "?" in uri and not '.' in uri:
                bookmark['uri'] = "about:blank"
            else:
                bookmark['uri'] = uri
            id += 1

            try:
                # This line messes up when the end of the file has been reached
                # Rather than coding properly, let's just catch the exception
                desc = link.parent.next_sibling.next_sibling
                if desc and desc.name == "dd":
                    bookmark["annos"] = []
                    annos = {}
                    annos["name"] = "bookmarkProperties/description"
                    annos["flags"] = 0
                    annos["expires"] = 4
                    annos["mimeType"] = ""
                    annos["type"] = 3
                    annos["value"] = desc.contents[0]
                    bookmark["annos"].append(annos)

Output :

IOError (Errno 27) : File too large

EDIT : Additionnal info : Python info :

$ python --version
Python 2.7.3

OSes used :

Linux Mint 13 : limit : 204.8kB

Debian 6.0 : limit : 768kB

Answer 1

This is not a Python error, but a limitation of the filesystem you're writing to, or an equivalent (artificial) restriction of your operating system.

Use ulimit -f to check your file size limit. It should say unlimited. If it doesn't, you will most likely want to edit /etc/security/limits.conf. You can use the following command to search for the problematic configuration:

grep fsize /etc/security/limits.conf /etc/security/limits.d/ -r

You may also want to check your filesystem quota or grsecurity limits.

Lastly, you may want to check mount to ensure that the filesystem you expect to be mounted is the one you see.