I tried using the y-combinator (in both Lua and Clojure) as I thought that would allow me to exceed the size of default stack implementations when using recursion. It seems I was mistaken. Yes, it works, but in both of these systems, the stack blows at exactly the same point as it does using plain old recursion. A lowish ~3600 in Clojure and a highish ~333000 on my Android Lua implementation. It is also a bit slower than regular recursion.
So is there anything to be gained by using the y-combinator, or is it just an intellectual exercise to prove a point? Have I missed something?
===
PS. Sorry, I should have made it clearer that I am aware I can use TCO to exceed the stack. My question does not relate to that. I am interested in this a) from the academic/intellectual point of view b) whether there is anything that can be done about those function that cannot be written tail recursively.
The Y combinator allows a non-recursive function to be used recursively, but that recursion still consumes stack space through nested function invocations.
For functions that can't be made tail-recursive, you could try refactoring them using continuation passing style, which would consume heap space instead of stack.
Here's a good overview of the topic: https://www.cs.cmu.edu/~15150/previous-semesters/2012-spring/resources/lectures/11.pdf