Search code examples
c++pythonboost-pythonpython-extensions

How to copy boost python list or his reference with "=" operator


take this code

#include<boost/python>
namespace bp = boost::python;

bp::list py_points; //initial list
other_class* C; // this class have a bp::list attribute called py_list

// ... some code ....

// in this part C.py_list.ptr() is 0x0

other_class->py_list = py_list; // problem here!!

the problem is with the operator "="

in the debugger in the object_core.hpp file, this is a boost python core file

inline api::object_base& api::object_base::operator=(api::object_base const& rhs)
{
    Py_INCREF(rhs.m_ptr);
    Py_DECREF(this->m_ptr); // in this line the program fail
    this->m_ptr = rhs.m_ptr;
    return *this;
}

what is the proper way to use the operator "="

edited

the problem is the stack, if the pointer other_class->py_list is null (or None because the class constructor is not called) the program can´t call the función Py_DECREF (Don't exist references before of the NULL pointer)

the problem is fix calling the constructor

other_class* C = new othe_class(); // fixed!!

Solution

  • The problem is not the assignment operator, the problem is that py_list's internal PyObject pointer is a nullptr. In the majority of cases, the pointer should not be null. From a Python point of view, it should at least manage a reference to the Python None object, as it done by a default constructed boost::python::object. The default constructor for boost::python::list creates a new empty list. Hence, the source of the problem likely resides within either other_class's constructor or the "some code" block.


    To elaborate on the question posed within the title, creating a reference or copying list in Boost.Python is the same as in Python:

    • The assignment operator will create a reference to a list.

      >>> a = ['spam']
      >>> b = a
      >>> b
      ['spam']
      >>> a is b
      True
      
    • One can slice a list to create a shallow copy.

      >>> a = ['spam']
      >>> b = a[:]
      >>> b
      ['spam']
      >>> a is b
      False
      

    Here is a complete example with the Python equivalent code annotated in the comments.

    #include <iostream>
    #include <boost/python.hpp>
    
    /// @brief Mockup class.
    struct other_class
    {
      boost::python::list py_list;
    };
    
    /// @brief Helper function to print object id and its string representation.
    std::string to_string(boost::python::object& o)
    {
      std::stringstream stream;
      stream << o.ptr() << " = "
             << boost::python::extract<std::string>(o.attr("__str__")())();
      return stream.str();
    }
    
    int main()
    {
      using std::cout;
      using std::endl;
      namespace python = boost::python;
    
      Py_Initialize();
      try
      {
        python::object object;                            // object = None
        cout << to_string(object) << "\n"                 // print object
             << "  is none check: " << object.is_none()   // print object is None
             << endl;
    
        // Create other_class and populate its list.
        other_class* c = new other_class();               // py_list = []
        cout << "c->py_list: " << to_string(c->py_list)   // print py_list
             << endl;
        c->py_list.append("spam");                        // py_list.append("spam")
        cout << "c->py_list: " << to_string(c->py_list)   // print py_list
             << endl;
    
        // Have list1 reference c->py_list.
        python::list list1;                                // list1 = []
        cout << "list1: " << to_string(list1) << "\n"      // print list1
             << "assign py_list to list1" << endl;
        list1 = c->py_list;                                // list1 = py_list
        cout << "list1: " << to_string(list1) << endl;     // print list1
    
        // Modify list1 and observe effects on pylist.
        cout << "modify list1" << endl;
        list1.append(42);                                 // list1.append(42)
        cout << "c->py_list: " << to_string(c->py_list)   // print py_list
             << endl;
    
        // Shallow-copy list1.
        cout << "copying list1 into list2" << endl;
        python::list list2(
            list1.slice(python::_, python::_));            // list2 = list1[:]
        list2.append("eggs");                              // list2.append("eggs")
        cout << "list2: " << to_string(list2) << "\n"      // print list2
             << "list1: " << to_string(list1) << endl;     // print list1
    
        delete c;
      }
      catch (python::error_already_set&)
      {
        PyErr_Print();
      }
    }
    

    Output:

    0x804e1ac = None
      is none check: 1
    c->py_list: 0xb707024c = []
    c->py_list: 0xb707024c = ['spam']
    list1: 0xb70da98c = []
    assign py_list to list1
    list1: 0xb707024c = ['spam']
    modify list1
    c->py_list: 0xb707024c = ['spam', 42]
    copying list1 into list2
    list2: 0xb707cb0c = ['spam', 42, 'eggs']
    list1: 0xb707024c = ['spam', 42]
    

    A few points to note in the output:

    • Default constructed boost::python::list objects manage a reference to an empty list. 0x804e1ac is None, and none of the list object's internal PyObject pointer manage a reference to it.
    • The list1 = py_list assignment causes list1 to manage a reference to the same list managed by py_list. This is exhibited in the output by list1 initially managing a reference to 0xb70da98c, but post-assignment, it manages a reference to 0xb707024c. With list1 and py_list managing the same list, a change to the list through one handle can be observed in the other handle.
    • Slicing constructed a new list. Thus, the PyObject internal point for list2 manages a different reference (0xb707cb0c) than list1's pointer (0xb707024c).