Search code examples
d

const immutable BigInt and range.join in D


I'm learning D and I have been playing with more and more functions and tools defined in phobos. I came across two functions that don't work when the parameters are const or immutable.

BigInt i = "42", j = "42";
writeln(i + j);
// This works, except when I add a const/immutable qualifier to i and j
// When const: main.d(23): Error: incompatible types for ((i) + (j)): 'const(BigInt)' and 'const(BigInt)'
// When immutable: main.d(23): Error: incompatible types for ((i) + (j)): 'immutable(BigInt)' and 'immutable(BigInt)'

The same happens with the std.array.join function.

int[] arr1 = [1, 2, 3, 4];
int[] arr2 = [5, 6];
writeln(join([arr1, arr2]));
// Again, the const and immutable errors are almost identical
// main.d(28): Error: template std.array.join(RoR, R)(RoR ror, R sep) if (isInputRange!RoR && isInputRange!(ElementType!RoR) && isInputRange!R && is(Unqual!(ElementType!(ElementType!RoR)) == Unqual!(ElementType!R))) cannot deduce template function from argument types !()(const(int[])[])

This is quite surprising to me. I have a C++ background so I usually write const everywhere, but it seems I can't do it in D.

As a D "user", I see that as a bug. Can someone explain me why this is not a bug and how I should call these functions with const/immutable data? Thanks.


Solution

  • First off, I should say that D's const is very different from C++'s const. Like C++, ideally you'd mark as much with it as possible, but unlike with C++, there are serious consequences to marking something const in D.

    In D, const is transitive, so it affects the entire type, not just the top level, and unlike in C++, you can't mutate it by casting it away or by using mutable (it's undefined behavior and will cause serious bugs if you try to cast away const from an object and then mutate it). The result of those two things is that there are many places where you just can't use const in D without making it impossible to do certain things.

    D's const provides real, solid guarantees that you can't mutate the object through that reference in any way, shape or form, whereas C++'s const just makes it so that you can't mutate anything which is const by accident, but you can easily cast away const and mutate the object (with defined behavior), or pieces of the object could be changed internally by const functions thanks to mutable. It's also trivial in C++ to return a mutable reference to the internals of a class from a const function even without casting or mutable (e.g. returning vector<int*> from a const function - the vector can't be mutated but everything it refers to can be). None of those are possible in D, as D guarantees full transitive const, and providing those guarantees makes it so that any circumstance where you need to get at something mutable from something const isn't going to work unless you create an entirely new copy of it.

    You should probably read over the answers to these questions:

    Logical const in D

    What is the difference between const and immutable in D?

    So, if you're slapping const on everything in D, you'll find that some things just won't work. Using const as much as you can is great for the same reasons that it is in C++, but the cost is much higher, so you have to be more restrictive about what you mark with const.

    Now, as to your specific issue here. BigInt is supposed to work with const and immutable but does not currently. There are a few open bugs on the issue. I believe that a lot of the problem stems from the fact that BigInt uses COW internally, and that doesn't play nicely with const or immutable. Fortunately, there's a pull request on github at the moment which fixes at least some of the problems, so I expect that BigInt will work with const and immutable in the near future, but for the moment, you can't.

    As for join, your example compiles just fine, so you copied your code wrong. There is no const in your example. Perhaps you meant

    const int[] arr1 = [1, 2, 3, 4];
    const int[] arr2 = [5, 6];
    writeln(join([arr1, arr2]));
    

    And that doesn't compile. And that's because you aren't passing a valid range of ranges to join. The type that you'd be passing to join in that case would be const(int[])[]. The outer array is mutable, so it's fine, but the inner ones - the ranges that you're trying to join together - are const, and nothing which is const can be a valid range, and that's because popFront won't work. For something to be a valid input range, this code must compile for it (and this is taken from inside of std.range.isInputRange).

        R r = void;       // can define a range object
        if (r.empty) {}   // can test for empty
        r.popFront();     // can invoke popFront()
        auto h = r.front; // can get the front of the range
    

    const(int[]) won't work with popFront as isInputRange requires. e.g.

    const int[] arr = [1, 2, 3];
    arr.popFront();
    

    won't compile, so isInputRange is false, and join won't compile with it.

    Now, fortunately, arrays are a bit special in that the compiler understands them, so the compiler knows that it's perfectly legit to turn const(int[]) into const(int)[] when you slice it. That is, it knows that giving you a tail-const slice won't be able to affect the original array (because the result is a new array, and while the elements are shared between the arrays, they're all const, so they still can't be mutated). So, the type of arr[] would be const(int)[] instead of const(int[]), and the type of [arr1[], arr2[]] is const(int)[][], which will work with join. So, you can do

    const int[] arr1 = [1, 2, 3, 4];
    const int[] arr2 = [5, 6];
    writeln(join([arr1[], arr2[]]));
    

    and your code will work just fine. However, that's just because you're using arrays. If you were dealing with user-defined ranges, the moment you made one of them const, you'd be stuck. This code won't compile

    const arr1 = filter!"true"([1, 2, 3, 4]);
    const arr2 = filter!"true"([5, 6]);
    writeln(join([arr1[], arr2[]]));
    

    And that's because the compiler doesn't know that it can safely get a tail-const slice from a ser-defined type. It needs to know that it can convert const MyRange!E to MyRange!(const E) and have the proper semantics. And it can't know that, because those are two different template instantiations, and they could have completely different internals. The programmer writing MyRange would have to be able to write opSlice such that it returns MyRange(const E) when the type is const MyRange!E or const MyRange!(const E), and that's actually hard to do (if nothing else, it very easily results in recursive template instantiations). Some clever use of static if and alias this should make it possible, but it's hard enough to do that pretty much no one does it right now. It's an open question as to how we're going to make it sane for user-defined types to make opSlice return a tail-const range. And until that question is solved, const and ranges just don't mix, because as soon as you get a const range, there's no way to get a tail-const slice of it which could have popFront called on it. So, once your range is const, it's const.

    Arrays are special, since they're built-in, so you can get away with using const with them as long as you slice them at the appropriate times (and the fact that templates are instantiated with their slice type rather than their original type helps), but in general, if you're using a range, just assume that you can't make it const. Hopefully, that changes someday, but for now, that's the way that it is.