How to make python not to create/copy pyobject from boost::shared_ptr during iteration on std::vector via boost::python?

Question

I have an object and std::vector of boost::shared_ptr-s of that objects in it which I want to transfer into python using boost::python. The problem here is the fact that during iteration over the exposed vector in python I'm getting different addresses every time for the same object. As I understand, python creates pyobject via my boost::shared_ptr on every iteration and after the iteration (actually after the second iteration) garbage collector frees that address, though the elements are boost::shared_ptr-s. When the elements are in python's list, they have their unique addresses which are not changing during iterations. This is what I'm trying to achieve. I tried to surf the internet by reaching different boost documentations and different QAs in stackoverflow but failed to get my issue resolved. I've created a small example demonstrating this issue. Here is the C++ code:

#include <vector>
#include <boost/shared_ptr.hpp>
#include <boost/make_shared.hpp>
#include <boost/python.hpp>

namespace bp = boost::python;

class A {
    public:
        A() : m_int(0) { }
        A(int numb) : m_int(numb) { }
        int get_m() const { return m_int; }
    private:
        int m_int;
};

std::ostream& operator<<(std::ostream& out, const A& obj) {
    out << obj.get_m();
    return out;
}

using APtr = boost::shared_ptr<A>;
using ACont = std::vector<APtr>;
using AContPtr = boost::shared_ptr<ACont>;

APtr CreateA() {
    return boost::make_shared<A>();
}

APtr CreateAFromNumb(int numb) {
    return boost::make_shared<A>(numb);
}

AContPtr CreateEmptyACont() {
    return boost::make_shared<ACont>();
}

AContPtr CreateFullACont(int size) {
    ACont vec{};
    for (int i = 0; i < size; ++i) {
        vec.push_back(boost::make_shared<A>(i));
    }
    return boost::make_shared<ACont>(vec);
}

BOOST_PYTHON_MODULE(example) {
    bp::class_<A, APtr, boost::noncopyable>("A", bp::no_init)
        .def("__init__", bp::make_constructor(CreateA))
        .def("__init__", bp::make_constructor(CreateAFromNumb))
        .def(bp::self_ns::str(bp::self))
        ;
    bp::register_ptr_to_python<APtr>();

    bp::class_<ACont, AContPtr, boost::noncopyable>("ACont", bp::no_init)
        .def("__init__", bp::make_constructor(CreateEmptyACont))
        .def("__init__", bp::make_constructor(CreateFullACont))
        .def("__iter__", bp::iterator<ACont>())
        .def("__len__", &ACont::size)
        ;
    bp::register_ptr_to_python<AContPtr>();
}

Here is the usage of it in python:

#!/usr/bin/env python

from example import *

vec = ACont(4)

print len(vec)
print "VEC", vec
for x in vec:
    print x,
    print repr(x)

print "VEC", vec
for x in vec:
    print x,
    print repr(x)

l = [x for x in vec]

print "LIST"
for x in l:
    print x,
    print repr(x)
print "LIST"
for x in l:
    print x,
    print repr(x)

When iterating over the exposed std::vector/boost::shared_ptr<std::vector> I'm getting different addresses for the same elements during just two sequential iterations.

1. Python creates pyobject on first iteration for the first item and shows it's address.
2. On the second iteration, python creates another pyobject for the second element on another address.
3. Garbage collector sees that the first address is no more usable and frees it.
4. On the third iteration, python creates another pyobject for the third element in the same address as the first one, since gc just freed it.
5. And so on.

But iteration over the list of the same elements is being done correctly, since python does not create/destruct pyobjects as they have references in the list itself. Here is the output of python run:

4
VEC <example.ACont object at 0x7ffff7ea9578>
0 <example.A object at 0x7ffff7eb27d0>    # address 1
1 <example.A object at 0x7ffff7eb2848>    # address 2
2 <example.A object at 0x7ffff7eb27d0>    # address 1 since gc just have freed it
3 <example.A object at 0x7ffff7eb2848>    # so on
VEC <example.ACont object at 0x7ffff7ea9578>
0 <example.A object at 0x7ffff7eb27d0>
1 <example.A object at 0x7ffff7eb2848>
2 <example.A object at 0x7ffff7eb27d0>
3 <example.A object at 0x7ffff7eb2848>
LIST
0 <example.A object at 0x7ffff7eb27d0>    # address 1
1 <example.A object at 0x7ffff7eb2848>    # address 2
2 <example.A object at 0x7ffff7eb28c0>    # address 3
3 <example.A object at 0x7ffff7eb2938>    # address 4
LIST
0 <example.A object at 0x7ffff7eb27d0>
1 <example.A object at 0x7ffff7eb2848>
2 <example.A object at 0x7ffff7eb28c0>
3 <example.A object at 0x7ffff7eb2938>

Summarizing, I would like to know whether there is a way not to create pyobjects in python for each element created in C++ taking into account a fact that these objects are boost::shared_ptr-s of some object. Thank you in advance.

P.S. I'm using boost version 1.60 and python 2.7 (anaconda package).

Answer 1

So, after long investigations, I found out that the task I wanted to solve does not have a real solution. This is due to the fact that python must register each object (created elsewhere, not in python) that just appeared in its environment to be aware of it. So the rest of scenario, regarding garbage collector, goes like described in the question. Neither boost/python nor swig could handle this case.
But, as a manual solution to this, to solve addressing issues I've created some kind of address mapping functionality. Since my issue was connected with object addresses, I transferred C++ address to python in order to see real object's address instead of python's one. Also, since I need hashing functionality working on addresses, I did the same thing there. Here is my update

// Same code as previously
std::string GetAddress(const APtr& a_ptr) {
    std::string repr{"<example.A object at "};
    std::stringstream sstream;
    sstream << std::hex << std::showbase << reinterpret_cast<std::uintptr_t>(a_ptr.get());
    repr += sstream.str() + ">";
    return repr;
}

static std::size_t Hash(const APtr& a_ptr) {
    std::hash<std::uintptr_t> hasher;
    return hasher(reinterpret_cast<std::uintptr_t>(a_ptr.get()));
}


BOOST_PYTHON_MODULE(example) {
    bp::class_<A, APtr, boost::noncopyable>("A", bp::no_init)
        .def("__init__", bp::make_constructor(CreateA))
        .def("__init__", bp::make_constructor(CreateAFromNumb))
        .def("__repr__", GetAddress)
        .def("__hash__", Hash)
        .def(bp::self_ns::str(bp::self))
        ;
    bp::register_ptr_to_python<APtr>();

    bp::class_<ACont, AContPtr, boost::noncopyable>("ACont", bp::no_init)
        .def("__init__", bp::make_constructor(CreateEmptyACont))
        .def("__init__", bp::make_constructor(CreateFullACont))
        .def("__iter__", bp::iterator<ACont>())
        .def("__len__", &ACont::size)
        ;
    bp::register_ptr_to_python<AContPtr>();
}

Anyway, thank you all for your effort trying to help. :)