Search code examples
pythonpython-2.7pyc

Disabling pyc files in production


pyc files have been the cause of grief in the past and I have just recently seen a few posts about preventing python from generating them. Currently we just run a script to clean them up after any code changes to ensure fresh ones get generated but it would be much easier to just disable them in general. Is there any side effects to disabling them in our production environment that maybe I am not aware of? What are the downsides of doing this?

The only real issue we experience is occasionally the files get out of date causing import errors and is difficult to debug after a massive refactor. Once realizing it is a pyc problem it is an easy enough fix, just run the script but that realization could come 30 minutes down the road of debugging.


Solution

  • Disabling the writing and use of compiled bytecode comes with a performance cost at startup -- your Python code is loaded into memory when the process spawns, and non-.pyc/.pyo files means the interpreter is forced to parse every imported file on startup. If you're currently removing all .pyc and .pyo files from your code directory and your imports, you're forcing a version of this already.

    I'm curious as to where they've caused grief in the past (are you disabling modified times on the filesystem?) as if the .py file is newer than the .pyc file, the Python interpreter (at least in common implementations, including CPython), will re-compile the source, but that's not very important for the scope of this question.

    If you're importing behind a function, e.g.:

    def my_database_call(budget_id):
        import some_expensive_module_to_import
        some_orm(...).get(budget_id)
    

    that import cost won't be incurred until that function is called (and perhaps, every subsequent time as well). This is not markedly different than if you were to allow bytecode (.pyc or "optimized" .pyo files), but at least in that case, you're paying to that import/parse/compile price only once.

    In the case you brought up in the comments, if you're "shelling out" to the OS to call a Python script, and are not allowing bytecode to be written in that call, every call incurs the compilation price.