Confusion regarding returning large values from functions and move

Question

I was watching an old panel discussion between Scott Mayers, Herb Sutter and Andrei Alexandrescu from C++ And Beyond 2011. In which to a question as to which c++11(c++0x at that time) feature people will get wrong, Andrei mentions that people assuming that move semantics will not involve a cost when returning large values from functions is wrong.Here's what he says

I will not design interfaces now to return big things by value, for the sheer reason that for all r-value references, there are going to be plenty of cases for which unnecessary copies are going to be created. And that shouldn't be forgotten.

I don't design and I don't condone designing interfaces that return big things by value, because some day someone is going to assign from it and assignment is going to be inefficient.

I don't think the cost of returning big things should be forgotten in the aftermath of victory of r-value references.

To which Herb elaborates the following:

I agree with you, but they are two different cases, one is you are generating a new result which you know you are going to put somewhere, and that's where you pass in by a non const reference, and have an out parameter, that's what out parameters are for.

There are other cases where you have two inputs and you are going to make something new, and there it is return by value, not as opposed to the out parameter thing, but its return by value instead of doing the clunky workaround that is error prone today, of heap allocation just to return a pointer, just to avoid that extra copy.

What is going on here, I just din't understand the point both these guys are making. What is the difference of cost of "assigning from it" Andrei was talking about? And Herb's explanation also, just bounced over my head. Can anyone please elaborate?

Also Consider the following code:

vector<BigData> GetVector(int someIndex)
{
   vector<BigData> toFill;
   // some processing
   // filling the vector
   return toFill;
}

I thought move semantics will make the above code equivalent to passing the empty vector as an out parameter. Isn't this so?

Here is a link to the video. The above points are made after a playing time of 41 mins or so.

Answer 1

I can't read the minds of Herb, Andrei or Scott. (Aside: none of these guys -- all very talented -- had anything at all to do with the Apollo space program (the background in their video)). However I can add some insight into rvalue-reference/move-semantics.

If you have a pure factory function like:

vector<BigData>
GetVector(int someIndex)
{
   vector<BigData> toFill;
   // some processing
   // filling the vector
   return toFill;
}

Then definitely return it by value. Every compiler today will do RVO, meaning the move/return from GetVector has exactly zero cost.

This being said, there is one context in which this is bad advice. For containers such as std::string and std::vector which have a notion of capacity(), or some other resource that adds performance while not impacting value, there can be examples where you don't want to throw away that resource gratuitously.

For example, consider:

vector<BigData> data;
while (I_need_to)
{
    data = GetVector(someIndex);
    process(data);
}

In this example, each time through the loop, GetVector is allocating a new vector, and this can be wasteful, because vector has a capacity that can be reused in a loop like this. Consider this rewrite:

void
GetVector(int someIndex, vector<BigData>& toFill)
{
   toFill.clear();
   // some processing
   // filling the vector
   return toFill;
}
// ...
vector<BigData> data;
while (I_need_to)
{
    GetVector(someIndex, data);
    process(data);
}

In this rewrite, data still gets a new value each time through the loop. But the difference is that the capacity() from the previous loop is saved, and reused for the current loop. If the required data.size() of this loop is less than the data.capacity() of the previous loop, then the current loop never needs to do a reallocation. And reducing trips to the heap are key to efficiency. Indeed, reducing trips to the heap is what move semantics is all about.

And I'm guessing this is the point discussed in the video.

I should emphasize: If the factory function isn't going to be used in a loop like this, or if the return type of the factory function can't make use of some previous resource (like capacity()), then the rewrite using an inOut parameter has no benefit.