Search code examples
azure-databricks

Databricks Exception: ERROR: maximum recursion depth exceeded in comparison

Has anyone come across the error the Databricks Error:

Exception: ERROR: maximum recursion depth exceeded in comparison

All I can say is that my Databricks configuration looks like the following

enter image description here

To me it looks like an internal error.

Any thoughts?

Solution

  • Error - Exception: ERROR: maximum recursion depth exceeded in comparison

    This error is related to Recursion Depth Limit in Python.

    This error says it all—maximum recursion depth exceeded in comparison. This tells you that Python’s recursion depth limit of 1000 is reached.

    A recursive function could call itself indefinitely. In other words, you could end up with an endless loop.

    Also, a stack overflow error can occur even if the recursion is not infinite. This can happen due to too big of a stack frame.

    In Python, the recursion depth limit takes these risks out of the equation. Python uses a maximum recursion depth of 1000 to ensure no stack overflow errors and infinite recursions are possible.

    This recursion limit is somewhat conservative, but it is reasonable as stack frames can become big in Python.

    How to Change the Recursion Depth Limit in Python

    import sys
print(sys.setrecursionlimit(2000))

    For more information follow this article by Artturi Jalli