In Closures, how is mutable captured by pointer or reference type stored in memory or handled in modern functional languages?

Question

I am writing a Transpiler for educational purpose. My transpiler transpires from my language to C language.
I am now writing closure syntax analyzer and code generation component.

I saw people saying that closures in C++ are actually transformed to unnamed structure types with captured values as variables inside.

Here is the reference.

This code

int c = 10;
auto closure = [=] () -> void {
    std::cout << c << std::endl;
};

is transformed into some sort of thing like this basically under the hood, so they say.

struct UNNAMED_TYPE_0 {
    int c;

    void operator() () const {
        std::cout << c << std::endl;
    }
};
// assume the closure is initialized and variables are assigned

If someone wants to mutate that int c when the closure executes, he/she has to pass this variable as ref [&c] () -> void { /* mutation comes here */}. But the problem is if we declare int c inside a function and create that closure inside the function like this

function<void()> aFunction() {
    int c = 10;
    auto closure = [&c] () -> void { c = 100; }
    return closure;
}

aFunction() ();

int c is captured but as soon as aFunction stack is destroyed, that int c is destroyed as well. This means, if we try to write on a deallocated address, we might run segmentation fault(core dumped) pointer error hopefully.

In Java,

// suppose this callback interface exists
public interface VoidCallback {
    public void method();
} 


public void aMethod() {
    int c = 10;
    VoidCallback callback = () -> c = 10; /* this gives an error */
    // local variables referenced from a lambda expression must be final or effectively final
}

Java handles closures like this and ensures there is no mutation to closure captures (let's say implicit captures). Meaning Java passes closure captures a copy rather than the ref. For reference or class types, only the Object pointer is passed as a copy. Although the pointer reference does not mutate, you can mutate the contents inside the object that pointer points to. This is basically the same as former one.

In Objective-C,

__block int c = 0;
// they say, this `int c` is allocated in heap instead of stack
// so that it exists until the closure is finished executing.
void (^ closure) (void) = ^void() {
    c = 10; // this is valid
    // this actually changed the `int c`'s value
}; 

In Swift

var a : Int = 10;
var closure = { [] () -> Void in
    a = 10; // this is valid by default
    // unless `var a` is declared as `let a`
};

So, this means, Objective-C and Swift allocate primitive capture lists as pointers. So that they can be mutated.

P.S: Please note that Swift closure capture list is only for a class or ref types but I'm meaning implicit captures over primitive types here.

This is

__block int c = 0;
// they say, this `int c` is allocated in heap instead of stack
// so that it exists until the closure is finished executing.
void (^ closure) (void) = ^void() {
    c = 10; // this is valid
    // this actually changed the `int c`'s value
}; 

the same as (basically) to this

int * c = malloc(sizeof(int));
*c = 0;
void (^ closure) (void) = ^void() {
    *c = 10;
    if (c) {
        free(c);
        c = NULL;
    }
}; 

Freeing the pointer variable as soon as the closure is done will be too bad I think.
What if there is a lot of closures that point to the variable and will mutate when executed?
What if those closures are passed around or got executed across different thread?

I came up with a solution using Reference Counting technique.
When a closure that mutate the variable is created, the variable will be retained.
When a closure that mutate the variable is destroyed, the variable will be released.
When there's no closure, the variable will be truly deallocated. To ensure thread safety, I would lock and unlock the counter variable address as the closures manipulate the reference counting technique.

If there is another technique, please guide me.
Any explanation in any language is greatly appreciated.

Currently, I have zero knowledge of assembly language.

For moderators, as this question is a kind of research, I beg you not to flag as too broad.

Answer 1

Following strikes me: "I am writing a Transpiler for educational purpose. My transpiler transpiles from my language to C language." Now, that means that the specification of your language defines how it is supposed to operate! We can't tell you how your language is supposed to operate.

Now, you already found a bunch of options:

• C++ doesn't do anything special to the local variable. If you keep a reference to it and use that when it ran out of scope, bad luck. That's the C++ spirit to not put any overhead on you but to allow you to shoot yourself in the foot if you don't pay attention.
• Java simply checks the code and tells you if you are trying to do anything it thinks is not guaranteed to be valid and gives you an error otherwise. It doesn't allow you to shoot yourself in the foot even if you want it badly.
• The other languages seem to convert the local variable with limited scope to a heap-based one. I'm not sure about their object model, but e.g. in Python you don't have anything similar to C++ local variables or Java primitive types at all, just as you don't have deterministic destructor calls (you gain similar things using with there, just for completeness), so this doesn't make any difference. These languages impose an overhead on you in order to guarantee that you don't have any dangling references (perhaps even if you don't really need it).

Now, the first thing is to decide which one best fits into your language's object model. Only then the question comes up how to best implement it. Concerning the implementation, there are a bunch of different approaches. Using a reference counter is one (implemented with lock-free, atomic operations though), using a linked list is another, or using a garbage collector.